2023-11-22 10:55:13,261 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-22 10:55:13,542 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-22 10:55:13,646 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-22 10:55:13,710 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-22 10:55:13,717 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-22 10:55:13,730 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=124, ProcessCount=171, AvailableMemoryMB=10444
2023-11-22 10:55:13,732 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-22 10:55:13,732 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-22 10:55:13,847 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1031864757:84 , srv1056123799:84 , srv1083289897:84 , srv1090162717:84 , srv1093019440:84 , srv1121507558:84 , srv1124827185:84 , srv1139930811:84 , srv1142887244:84 , srv1147042128:84 , srv1189436831:84 , srv1225454781:84 , srv1228144214:84 , srv1234464157:84 , srv1257308800:84 , srv1334252772:84 , srv1339217732:84 , srv1348947982:84 , srv1382746480:84 , srv1384713577:84 , srv1437625610:84 , srv143848657:84 , srv1439988115:84 , srv1483650320:84 , srv1485309442:84 , srv1502618330:84 , srv1502757227:84 , srv1519048652:84 , srv1544219565:84 , srv1561063077:84 , srv161574161:84 , srv1628260843:84 , srv1634707522:84 , srv1639563251:84 , srv1654555212:84 , srv1657444978:84 , srv1662080911:84 , srv1689778518:84 , srv1769235123:84 , srv1771748198:84 , srv1775700432:84 , srv1818298472:84 , srv1825223088:84 , srv1881534370:84 , srv188434041:84 , srv1893040392:84 , srv1893348540:84 , srv1941366190:84 , srv1982870054:84 , srv1990164821:84 , srv2012391178:84 , srv2053869356:84 , srv2065363333:84 , srv2083344575:84 , srv2098372719:84 , srv2124413086:84 , srv2145258446:84 , srv219479679:84 , srv224810069:84 , srv246626496:84 , srv281712466:84 , srv286375581:84 , srv316471625:84 , srv352211921:84 , srv358511245:84 , srv359676453:84 , srv397520776:84 , srv416562954:84 , srv428122473:84 , srv438929263:84 , srv456701703:84 , srv469610813:84 , srv515915299:84 , srv518501139:84 , srv531658814:84 , srv544748336:84 , srv547688756:84 , srv56066927:84 , srv611990865:84 , srv619995198:84 , srv623763091:84 , srv691841658:84 , srv692785501:84 , srv708193963:84 , srv709167492:84 , srv735270990:84 , srv744811492:84 , srv777356594:84 , srv828873373:84 , srv854081264:84 , srv860766308:84 , srv871250717:84 , srv931701891:84 , srv932750181:84 , srv934516854:84 , srv951414050:84 , srv971046738:84 , srv983711203:84 , srv987115288:84 , srv766353056:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-22 10:55:13,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:13,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:13,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:13,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:13,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:13,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:13,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:13,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:13,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:13,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:13,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:13,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:13,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:14,249 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:14,250 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-22 10:55:14,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-22 10:55:14,631 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 673 ms to try 5000 different iterations.  Found a solution that moves 4702 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17846733150765592. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04018786163099039, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05524966291775229, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5224444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04018786163099039, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:14,648 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1437625610:81 , srv2098372719:82 , srv1031864757:83 , srv1439988115:83 , srv1561063077:83 , srv1662080911:83 , srv1893348540:83 , srv1990164821:83 , srv224810069:83 , srv352211921:83 , srv438929263:83 , srv456701703:83 , srv531658814:83 , srv828873373:83 , srv951414050:83 , srv1121507558:84 , srv1339217732:84 , srv1689778518:84 , srv1881534370:84 , srv1893040392:84 , srv219479679:84 , srv316471625:84 , srv515915299:84 , srv777356594:84 , srv987115288:84 , srv1093019440:85 , srv1142887244:85 , srv1147042128:85 , srv1189436831:85 , srv1382746480:85 , srv143848657:85 , srv1769235123:85 , srv1771748198:85 , srv1982870054:85 , srv469610813:85 , srv691841658:85 , srv709167492:85 , srv931701891:85 , srv971046738:85 , srv1083289897:86 , srv1257308800:86 , srv1348947982:86 , srv1639563251:86 , srv1825223088:86 , srv2065363333:86 , srv246626496:86 , srv428122473:86 , srv547688756:86 , srv56066927:86 , srv623763091:86 , srv735270990:86 , srv871250717:86 , srv1124827185:87 , srv1225454781:87 , srv1228144214:87 , srv1234464157:87 , srv1483650320:87 , srv1502757227:87 , srv161574161:87 , srv1628260843:87 , srv1775700432:87 , srv2012391178:87 , srv286375581:87 , srv359676453:87 , srv544748336:87 , srv611990865:87 , srv619995198:87 , srv692785501:87 , srv854081264:87 , srv860766308:87 , srv1056123799:88 , srv1485309442:88 , srv1502618330:88 , srv1544219565:88 , srv1634707522:88 , srv1654555212:88 , srv1818298472:88 , srv2053869356:88 , srv2083344575:88 , srv2145258446:88 , srv397520776:88 , srv744811492:88 , srv932750181:88 , srv1519048652:89 , srv188434041:89 , srv2124413086:89 , srv281712466:89 , srv358511245:89 , srv934516854:89 , srv983711203:89 , srv1139930811:90 , srv1657444978:90 , srv1941366190:90 , srv416562954:90 , srv518501139:90 , srv708193963:90 , srv1090162717:95 , srv1334252772:96 , srv1384713577:99 , srv766353056:447 }
2023-11-22 10:55:14,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:14,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:14,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:14,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:14,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:14,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:14,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:14,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:14,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:14,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:14,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04018786163099039, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05524966291775229, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04018786163099039, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:14,798 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-22 10:55:14,799 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17843440739995486, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04018786163099039, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05524966291775229, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04018786163099039, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-22 10:55:15,053 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 352 ms to try 5000 different iterations.  Found a solution that moves 2683 regions; Going from a computed imbalance of 0.17843440739995486 to a new imbalance of 0.01624037235950876. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037470527138544554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2981111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:15,063 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1384713577:73 , srv1090162717:74 , srv1334252772:78 , srv1139930811:80 , srv281712466:81 , srv708193963:81 , srv2124413086:82 , srv416562954:82 , srv983711203:82 , srv1485309442:83 , srv161574161:83 , srv1654555212:83 , srv1941366190:83 , srv2083344575:83 , srv2145258446:83 , srv744811492:83 , srv854081264:83 , srv934516854:83 , srv1225454781:84 , srv1502757227:84 , srv1519048652:84 , srv1657444978:84 , srv188434041:84 , srv2012391178:84 , srv2053869356:84 , srv397520776:84 , srv544748336:84 , srv547688756:84 , srv619995198:84 , srv932750181:84 , srv1056123799:85 , srv1093019440:85 , srv1257308800:85 , srv1348947982:85 , srv1502618330:85 , srv1544219565:85 , srv1628260843:85 , srv1634707522:85 , srv1639563251:85 , srv1775700432:85 , srv1825223088:85 , srv1982870054:85 , srv286375581:85 , srv358511245:85 , srv428122473:85 , srv518501139:85 , srv692785501:85 , srv735270990:85 , srv777356594:85 , srv871250717:85 , srv1124827185:86 , srv1189436831:86 , srv1228144214:86 , srv1483650320:86 , srv1771748198:86 , srv1818298472:86 , srv1893040392:86 , srv246626496:86 , srv359676453:86 , srv611990865:86 , srv623763091:86 , srv691841658:86 , srv971046738:86 , srv1121507558:87 , srv1234464157:87 , srv1382746480:87 , srv143848657:87 , srv1689778518:87 , srv1881534370:87 , srv1893348540:87 , srv2065363333:87 , srv219479679:87 , srv456701703:87 , srv709167492:87 , srv860766308:87 , srv1083289897:88 , srv1142887244:88 , srv1339217732:88 , srv1662080911:88 , srv1990164821:88 , srv316471625:88 , srv438929263:88 , srv531658814:88 , srv56066927:88 , srv951414050:88 , srv1147042128:89 , srv1561063077:89 , srv1769235123:89 , srv469610813:89 , srv515915299:89 , srv828873373:89 , srv1031864757:90 , srv1439988115:90 , srv2098372719:90 , srv352211921:90 , srv931701891:90 , srv987115288:90 , srv224810069:91 , srv1437625610:94 , srv766353056:531 }
2023-11-22 10:55:15,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:15,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:15,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:15,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:15,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:15,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:15,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037470527138544554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:15,226 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-22 10:55:15,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016221585591971125, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037470527138544554, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-22 10:55:15,450 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 324 ms to try 5000 different iterations.  Found a solution that moves 967 regions; Going from a computed imbalance of 0.016221585591971125 to a new imbalance of 6.774586571650432E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008126693517330957); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011967032904743342); MoveCostFunction : (multiplier=7.0, imbalance=0.10744444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008126693517330957); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:15,451 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1437625610:81 , srv931701891:82 , srv1147042128:83 , srv1234464157:83 , srv1334252772:83 , srv1634707522:83 , srv1818298472:83 , srv188434041:83 , srv224810069:83 , srv358511245:83 , srv515915299:83 , srv518501139:83 , srv611990865:83 , srv1056123799:84 , srv1083289897:84 , srv1142887244:84 , srv1339217732:84 , srv1439988115:84 , srv1544219565:84 , srv1628260843:84 , srv1657444978:84 , srv1769235123:84 , srv1941366190:84 , srv2065363333:84 , srv359676453:84 , srv469610813:84 , srv860766308:84 , srv1031864757:85 , srv1124827185:85 , srv1228144214:85 , srv1384713577:85 , srv1485309442:85 , srv1502618330:85 , srv1519048652:85 , srv1561063077:85 , srv1662080911:85 , srv1771748198:85 , srv1775700432:85 , srv2098372719:85 , srv2124413086:85 , srv2145258446:85 , srv246626496:85 , srv286375581:85 , srv352211921:85 , srv397520776:85 , srv416562954:85 , srv56066927:85 , srv623763091:85 , srv709167492:85 , srv828873373:85 , srv934516854:85 , srv1225454781:86 , srv1348947982:86 , srv1382746480:86 , srv143848657:86 , srv1483650320:86 , srv1639563251:86 , srv1689778518:86 , srv1825223088:86 , srv1881534370:86 , srv1990164821:86 , srv2012391178:86 , srv2053869356:86 , srv2083344575:86 , srv281712466:86 , srv316471625:86 , srv428122473:86 , srv438929263:86 , srv531658814:86 , srv544748336:86 , srv619995198:86 , srv691841658:86 , srv692785501:86 , srv708193963:86 , srv735270990:86 , srv932750181:86 , srv951414050:86 , srv971046738:86 , srv983711203:86 , srv987115288:86 , srv1093019440:87 , srv1121507558:87 , srv1139930811:87 , srv1189436831:87 , srv1257308800:87 , srv1893348540:87 , srv219479679:87 , srv547688756:87 , srv744811492:87 , srv871250717:87 , srv1090162717:88 , srv1502757227:88 , srv161574161:88 , srv1654555212:88 , srv1893040392:88 , srv1982870054:88 , srv456701703:88 , srv777356594:88 , srv854081264:88 , srv766353056:552 }
2023-11-22 10:55:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:15,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:15,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:15,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:15,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:15,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:15,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:15,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:15,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:15,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:15,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:15,584 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008126693517330957); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011967032904743342); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008126693517330957); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:15,584 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-22 10:55:15,584 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.706875784438759E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008126693517330957); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011967032904743342); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008126693517330957); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-22 10:55:15,909 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 422 ms to try 5000 different iterations.  Found a solution that moves 289 regions; Going from a computed imbalance of 6.706875784438759E-4 to a new imbalance of 5.80908767396155E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03211111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:15,910 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv987115288:82 , srv1483650320:83 , srv1654555212:83 , srv2053869356:83 , srv56066927:83 , srv1031864757:84 , srv1083289897:84 , srv1121507558:84 , srv1124827185:84 , srv1225454781:84 , srv1228144214:84 , srv1257308800:84 , srv143848657:84 , srv1502618330:84 , srv1502757227:84 , srv1519048652:84 , srv1544219565:84 , srv161574161:84 , srv1639563251:84 , srv1657444978:84 , srv1769235123:84 , srv1881534370:84 , srv1893040392:84 , srv1982870054:84 , srv316471625:84 , srv352211921:84 , srv416562954:84 , srv456701703:84 , srv469610813:84 , srv518501139:84 , srv619995198:84 , srv692785501:84 , srv744811492:84 , srv854081264:84 , srv860766308:84 , srv871250717:84 , srv932750181:84 , srv934516854:84 , srv1056123799:85 , srv1090162717:85 , srv1139930811:85 , srv1142887244:85 , srv1147042128:85 , srv1189436831:85 , srv1234464157:85 , srv1334252772:85 , srv1348947982:85 , srv1382746480:85 , srv1384713577:85 , srv1437625610:85 , srv1439988115:85 , srv1561063077:85 , srv1628260843:85 , srv1689778518:85 , srv1771748198:85 , srv1775700432:85 , srv1818298472:85 , srv1825223088:85 , srv1893348540:85 , srv1941366190:85 , srv1990164821:85 , srv2012391178:85 , srv2065363333:85 , srv2083344575:85 , srv219479679:85 , srv224810069:85 , srv246626496:85 , srv286375581:85 , srv358511245:85 , srv359676453:85 , srv397520776:85 , srv428122473:85 , srv531658814:85 , srv544748336:85 , srv547688756:85 , srv708193963:85 , srv709167492:85 , srv735270990:85 , srv777356594:85 , srv828873373:85 , srv931701891:85 , srv971046738:85 , srv983711203:85 , srv1093019440:86 , srv1339217732:86 , srv1485309442:86 , srv1634707522:86 , srv1662080911:86 , srv188434041:86 , srv2098372719:86 , srv2124413086:86 , srv2145258446:86 , srv281712466:86 , srv438929263:86 , srv611990865:86 , srv623763091:86 , srv691841658:86 , srv951414050:86 , srv515915299:87 , srv766353056:612 }
2023-11-22 10:55:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:15,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:15,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:15,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:15,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:15,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:15,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:15,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:15,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:15,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:15,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:15,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:16,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:16,042 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-22 10:55:16,042 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.606725548789122E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-22 10:55:16,282 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 340 ms to try 5000 different iterations.  Found a solution that moves 40 regions; Going from a computed imbalance of 5.606725548789122E-5 to a new imbalance of 5.9225789799244974E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0044444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:16,283 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1093019440:83 , srv1189436831:83 , srv1382746480:83 , srv1502757227:83 , srv2098372719:83 , srv219479679:83 , srv438929263:83 , srv515915299:83 , srv623763091:83 , srv691841658:83 , srv951414050:83 , srv1031864757:84 , srv1056123799:84 , srv1083289897:84 , srv1090162717:84 , srv1121507558:84 , srv1124827185:84 , srv1139930811:84 , srv1142887244:84 , srv1147042128:84 , srv1228144214:84 , srv1234464157:84 , srv1257308800:84 , srv1334252772:84 , srv1339217732:84 , srv1348947982:84 , srv1384713577:84 , srv143848657:84 , srv1439988115:84 , srv1483650320:84 , srv1485309442:84 , srv1502618330:84 , srv1519048652:84 , srv1561063077:84 , srv161574161:84 , srv1634707522:84 , srv1654555212:84 , srv1657444978:84 , srv1662080911:84 , srv1689778518:84 , srv1769235123:84 , srv1775700432:84 , srv1818298472:84 , srv1825223088:84 , srv188434041:84 , srv1893040392:84 , srv1893348540:84 , srv1941366190:84 , srv1982870054:84 , srv1990164821:84 , srv2012391178:84 , srv2065363333:84 , srv2083344575:84 , srv2124413086:84 , srv2145258446:84 , srv224810069:84 , srv246626496:84 , srv281712466:84 , srv286375581:84 , srv316471625:84 , srv352211921:84 , srv358511245:84 , srv359676453:84 , srv397520776:84 , srv428122473:84 , srv456701703:84 , srv469610813:84 , srv518501139:84 , srv531658814:84 , srv544748336:84 , srv547688756:84 , srv56066927:84 , srv611990865:84 , srv692785501:84 , srv708193963:84 , srv709167492:84 , srv735270990:84 , srv744811492:84 , srv777356594:84 , srv828873373:84 , srv854081264:84 , srv860766308:84 , srv871250717:84 , srv931701891:84 , srv932750181:84 , srv971046738:84 , srv983711203:84 , srv987115288:84 , srv1225454781:85 , srv1437625610:85 , srv1544219565:85 , srv1628260843:85 , srv1639563251:85 , srv1771748198:85 , srv1881534370:85 , srv2053869356:85 , srv416562954:85 , srv619995198:85 , srv934516854:85 , srv766353056:684 }
2023-11-22 10:55:16,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:16,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1689778518=37, srv56066927=77, srv316471625=62, srv1142887244=8, srv1502618330=25, srv428122473=68, srv219479679=57, srv469610813=71, srv1893040392=45, srv246626496=59, srv691841658=81, srv1639563251=33, srv951414050=96, srv1225454781=11, srv456701703=70, srv358511245=64, srv547688756=76, srv860766308=91, srv1083289897=2, srv1990164821=49, srv286375581=61, srv1031864757=0, srv1485309442=24, srv2065363333=52, srv1483650320=23, srv2124413086=55, srv1544219565=28, srv1384713577=19, srv438929263=69, srv766353056=87, srv1437625610=20, srv708193963=83, srv1439988115=22, srv416562954=67, srv1228144214=12, srv1769235123=38, srv1234464157=13, srv2053869356=51, srv1662080911=36, srv1982870054=48, srv281712466=60, srv544748336=75, srv777356594=88, srv971046738=97, srv692785501=82, srv1339217732=16, srv1654555212=34, srv1628260843=31, srv1893348540=46, srv1382746480=18, srv1519048652=27, srv1657444978=35, srv987115288=99, srv1941366190=47, srv1189436831=10, srv1825223088=42, srv623763091=80, srv1771748198=39, srv1818298472=41, srv735270990=85, srv359676453=65, srv161574161=30, srv871250717=92, srv744811492=86, srv1334252772=15, srv2083344575=53, srv709167492=84, srv531658814=74, srv515915299=72, srv1139930811=7, srv1502757227=26, srv611990865=78, srv932750181=94, srv1348947982=17, srv1775700432=40, srv1093019440=4, srv1056123799=1, srv1121507558=5, srv1090162717=3, srv188434041=44, srv854081264=90, srv143848657=21, srv352211921=63, srv2098372719=54, srv1124827185=6, srv934516854=95, srv2012391178=50, srv1881534370=43, srv828873373=89, srv983711203=98, srv931701891=93, srv619995198=79, srv2145258446=56, srv224810069=58, srv518501139=73, srv1634707522=32, srv1147042128=9, srv1561063077=29, srv1257308800=14, srv397520776=66} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:16,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:16,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:16,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:16,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-22 10:55:16,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-22 10:55:16,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-22 10:55:16,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-22 10:55:16,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-22 10:55:16,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-22 10:55:16,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-22 10:55:16,420 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=5.642492993526674E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:16,421 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1093019440:83 , srv1189436831:83 , srv1382746480:83 , srv1502757227:83 , srv2098372719:83 , srv219479679:83 , srv438929263:83 , srv515915299:83 , srv623763091:83 , srv691841658:83 , srv951414050:83 , srv1031864757:84 , srv1056123799:84 , srv1083289897:84 , srv1090162717:84 , srv1121507558:84 , srv1124827185:84 , srv1139930811:84 , srv1142887244:84 , srv1147042128:84 , srv1228144214:84 , srv1234464157:84 , srv1257308800:84 , srv1334252772:84 , srv1339217732:84 , srv1348947982:84 , srv1384713577:84 , srv143848657:84 , srv1439988115:84 , srv1483650320:84 , srv1485309442:84 , srv1502618330:84 , srv1519048652:84 , srv1561063077:84 , srv161574161:84 , srv1634707522:84 , srv1654555212:84 , srv1657444978:84 , srv1662080911:84 , srv1689778518:84 , srv1769235123:84 , srv1775700432:84 , srv1818298472:84 , srv1825223088:84 , srv188434041:84 , srv1893040392:84 , srv1893348540:84 , srv1941366190:84 , srv1982870054:84 , srv1990164821:84 , srv2012391178:84 , srv2065363333:84 , srv2083344575:84 , srv2124413086:84 , srv2145258446:84 , srv224810069:84 , srv246626496:84 , srv281712466:84 , srv286375581:84 , srv316471625:84 , srv352211921:84 , srv358511245:84 , srv359676453:84 , srv397520776:84 , srv428122473:84 , srv456701703:84 , srv469610813:84 , srv518501139:84 , srv531658814:84 , srv544748336:84 , srv547688756:84 , srv56066927:84 , srv611990865:84 , srv692785501:84 , srv708193963:84 , srv709167492:84 , srv735270990:84 , srv744811492:84 , srv777356594:84 , srv828873373:84 , srv854081264:84 , srv860766308:84 , srv871250717:84 , srv931701891:84 , srv932750181:84 , srv971046738:84 , srv983711203:84 , srv987115288:84 , srv1225454781:85 , srv1437625610:85 , srv1544219565:85 , srv1628260843:85 , srv1639563251:85 , srv1771748198:85 , srv1881534370:85 , srv2053869356:85 , srv416562954:85 , srv619995198:85 , srv934516854:85 , srv766353056:684 }
2023-11-22 10:55:16,446 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=194 (was 124) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=9960 (was 10444)
2023-11-22 10:55:16,453 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=194, ProcessCount=171, AvailableMemoryMB=9960
2023-11-22 10:55:16,453 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-22 10:55:16,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-22 10:55:16,464 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047419398:84 , srv1063037308:84 , srv1165825130:84 , srv1207246530:84 , srv1225426071:84 , srv1263223405:84 , srv1326332926:84 , srv1349784255:84 , srv1365438821:84 , srv151873073:84 , srv1519890058:84 , srv1545093042:84 , srv1547289266:84 , srv1593775800:84 , srv175712117:84 , srv2001337894:84 , srv410520084:84 , srv435283802:84 , srv497945:84 , srv51703623:84 , srv541855558:84 , srv648411710:84 , srv729600980:84 , srv736044387:84 , srv749690497:84 , srv760330667:84 , srv765118875:84 , srv806126833:84 , srv969725315:84 , srv1510540896:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-22 10:55:16,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:55:16,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1326332926=6, srv497945=19, srv1349784255=7, srv435283802=18, srv760330667=26, srv1063037308=1, srv1047419398=0, srv541855558=21, srv51703623=20, srv1165825130=2, srv151873073=10, srv749690497=25, srv2001337894=16, srv765118875=27, srv806126833=28, srv1225426071=4, srv1519890058=11, srv1545093042=12, srv410520084=17, srv648411710=22, srv1547289266=13, srv1263223405=5, srv1593775800=14, srv175712117=15, srv969725315=29, srv729600980=23, srv1365438821=8, srv1207246530=3, srv736044387=24, srv1510540896=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:55:16,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:55:16,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:55:16,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:55:16,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:55:16,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:55:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:55:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:55:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:55:16,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-22 10:55:16,489 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1173736606971432, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:55:16,489 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912562883836534, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1173736606971432, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-22 10:57:16,477 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6038500 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912562883836534 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:57:16,484 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1047419398:90 , srv1063037308:90 , srv1165825130:90 , srv1207246530:90 , srv1225426071:90 , srv1263223405:90 , srv1326332926:90 , srv1349784255:90 , srv1365438821:90 , srv1510540896:90 , srv151873073:90 , srv1519890058:90 , srv1545093042:90 , srv1547289266:90 , srv1593775800:90 , srv175712117:90 , srv2001337894:90 , srv410520084:90 , srv435283802:90 , srv497945:90 , srv51703623:90 , srv541855558:90 , srv648411710:90 , srv729600980:90 , srv736044387:90 , srv749690497:90 , srv760330667:90 , srv765118875:90 , srv806126833:90 , srv969725315:90 }
2023-11-22 10:57:16,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-22 10:57:16,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1326332926=6, srv497945=19, srv1349784255=7, srv435283802=18, srv760330667=26, srv1063037308=1, srv1047419398=0, srv541855558=21, srv51703623=20, srv1165825130=2, srv151873073=10, srv749690497=25, srv2001337894=16, srv765118875=27, srv806126833=28, srv1225426071=4, srv1519890058=11, srv1545093042=12, srv410520084=17, srv648411710=22, srv1547289266=13, srv1263223405=5, srv1593775800=14, srv175712117=15, srv969725315=29, srv729600980=23, srv1365438821=8, srv1207246530=3, srv736044387=24, srv1510540896=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-22 10:57:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-22 10:57:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-22 10:57:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-22 10:57:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-22 10:57:16,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-22 10:57:16,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-22 10:57:16,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-22 10:57:16,502 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-22 10:57:16,502 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1047419398:90 , srv1063037308:90 , srv1165825130:90 , srv1207246530:90 , srv1225426071:90 , srv1263223405:90 , srv1326332926:90 , srv1349784255:90 , srv1365438821:90 , srv1510540896:90 , srv151873073:90 , srv1519890058:90 , srv1545093042:90 , srv1547289266:90 , srv1593775800:90 , srv175712117:90 , srv2001337894:90 , srv410520084:90 , srv435283802:90 , srv497945:90 , srv51703623:90 , srv541855558:90 , srv648411710:90 , srv729600980:90 , srv736044387:90 , srv749690497:90 , srv760330667:90 , srv765118875:90 , srv806126833:90 , srv969725315:90 }
2023-11-22 10:57:16,514 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=261 (was 194) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 171), AvailableMemoryMB=9846 (was 9960)