2023-04-23 02:55:45,428 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-23 02:55:45,740 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-23 02:55:45,854 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-23 02:55:45,924 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 02:55:45,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-23 02:55:45,950 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=170, AvailableMemoryMB=13202
2023-04-23 02:55:45,952 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 02:55:45,953 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-23 02:55:46,099 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002946250:84 , srv1014030431:84 , srv1023792292:84 , srv1040088768:84 , srv1060603846:84 , srv1066459317:84 , srv1071964243:84 , srv1086022864:84 , srv1134830110:84 , srv1146320808:84 , srv1175929407:84 , srv1197322044:84 , srv1219323967:84 , srv1229358843:84 , srv1234295313:84 , srv1247369979:84 , srv1250733788:84 , srv1278229023:84 , srv127939995:84 , srv1285707754:84 , srv1319360500:84 , srv1325591250:84 , srv133334182:84 , srv1338367680:84 , srv135186097:84 , srv1355442980:84 , srv1359889875:84 , srv1368419522:84 , srv1394722626:84 , srv1424716894:84 , srv1427154718:84 , srv1450074241:84 , srv1454873234:84 , srv1459931545:84 , srv1472013188:84 , srv151089613:84 , srv1568292427:84 , srv159426889:84 , srv1629009354:84 , srv1659873288:84 , srv1660505264:84 , srv1686364927:84 , srv1689128018:84 , srv173429722:84 , srv1738742984:84 , srv1741967321:84 , srv1758448226:84 , srv1762183668:84 , srv1793592523:84 , srv1821442856:84 , srv1853829030:84 , srv1885252248:84 , srv188924944:84 , srv195357215:84 , srv1956171458:84 , srv201202780:84 , srv2026853474:84 , srv2093519387:84 , srv2107369397:84 , srv2109679890:84 , srv2126511672:84 , srv2134695562:84 , srv288170821:84 , srv297695370:84 , srv363863194:84 , srv366675842:84 , srv380368208:84 , srv418306223:84 , srv420554322:84 , srv426920810:84 , srv430439604:84 , srv458644154:84 , srv497083787:84 , srv511221442:84 , srv514511688:84 , srv52766823:84 , srv546688633:84 , srv581545308:84 , srv589232211:84 , srv609069895:84 , srv612621212:84 , srv658499282:84 , srv665773888:84 , srv672289552:84 , srv677914475:84 , srv694730216:84 , srv711285756:84 , srv718754493:84 , srv724528861:84 , srv728709139:84 , srv731460424:84 , srv757828531:84 , srv798380484:84 , srv850412696:84 , srv852360074:84 , srv878370535:84 , srv880346403:84 , srv885195855:84 , srv89477090:84 , srv25557100:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-23 02:55:46,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:46,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv798380484=93, srv1175929407=10, srv366675842=66, srv1219323967=12, srv363863194=65, srv1197322044=11, srv135186097=24, srv850412696=94, srv1325591250=21, srv1023792292=2, srv1060603846=4, srv297695370=64, srv1758448226=46, srv188924944=52, srv426920810=70, srv731460424=91, srv497083787=73, srv885195855=98, srv1472013188=34, srv173429722=43, srv1285707754=19, srv514511688=75, srv1762183668=47, srv2107369397=58, srv133334182=22, srv1229358843=13, srv1629009354=38, srv1454873234=32, srv1821442856=49, srv458644154=72, srv2134695562=61, srv1002946250=0, srv1853829030=50, srv1568292427=36, srv159426889=37, srv1659873288=39, srv672289552=84, srv1660505264=40, srv418306223=68, srv1689128018=42, srv1086022864=7, srv1427154718=30, srv420554322=69, srv724528861=89, srv430439604=71, srv1359889875=26, srv2093519387=57, srv1885252248=51, srv1368419522=27, srv546688633=77, srv1686364927=41, srv1793592523=48, srv2126511672=60, srv609069895=80, srv878370535=96, srv711285756=87, srv2026853474=56, srv728709139=90, srv1741967321=45, srv1247369979=15, srv1338367680=23, srv852360074=95, srv1040088768=3, srv1450074241=31, srv511221442=74, srv677914475=85, srv201202780=55, srv195357215=53, srv1250733788=16, srv1319360500=20, srv665773888=83, srv89477090=99, srv658499282=82, srv1394722626=28, srv288170821=63, srv1066459317=5, srv127939995=18, srv612621212=81, srv1071964243=6, srv52766823=76, srv1134830110=8, srv1738742984=44, srv380368208=67, srv880346403=97, srv151089613=35, srv1459931545=33, srv2109679890=59, srv25557100=62, srv757828531=92, srv589232211=79, srv1278229023=17, srv694730216=86, srv1014030431=1, srv1424716894=29, srv1355442980=25, srv718754493=88, srv1956171458=54, srv1234295313=14, srv1146320808=9, srv581545308=78} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:46,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:46,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:46,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:46,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:46,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 02:55:46,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 02:55:46,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 02:55:46,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 02:55:46,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 02:55:46,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 02:55:46,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 02:55:46,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 02:55:46,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 02:55:46,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 02:55:46,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 02:55:46,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 02:55:46,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 02:55:46,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 02:55:46,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 02:55:46,514 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:46,514 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 02:55:46,515 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 02:55:46,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 589 ms to try 5000 different iterations.  Found a solution that moves 4694 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1777890420761083. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038498116845159046, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05971875724033418, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5215555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038498116845159046, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:46,883 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1821442856:80 , srv1229358843:82 , srv1146320808:83 , srv1278229023:83 , srv1325591250:83 , srv1450074241:83 , srv1454873234:83 , srv1568292427:83 , srv1741967321:83 , srv2107369397:83 , srv2134695562:83 , srv546688633:83 , srv1002946250:84 , srv1368419522:84 , srv173429722:84 , srv1738742984:84 , srv201202780:84 , srv430439604:84 , srv458644154:84 , srv718754493:84 , srv728709139:84 , srv1040088768:85 , srv1219323967:85 , srv1250733788:85 , srv1338367680:85 , srv1424716894:85 , srv1427154718:85 , srv1686364927:85 , srv1793592523:85 , srv195357215:85 , srv2109679890:85 , srv366675842:85 , srv612621212:85 , srv658499282:85 , srv731460424:85 , srv850412696:85 , srv880346403:85 , srv1175929407:86 , srv1285707754:86 , srv1319360500:86 , srv1355442980:86 , srv151089613:86 , srv159426889:86 , srv1660505264:86 , srv1762183668:86 , srv188924944:86 , srv297695370:86 , srv363863194:86 , srv514511688:86 , srv589232211:86 , srv672289552:86 , srv677914475:86 , srv711285756:86 , srv724528861:86 , srv89477090:86 , srv1014030431:87 , srv1134830110:87 , srv1197322044:87 , srv1247369979:87 , srv1359889875:87 , srv1689128018:87 , srv1853829030:87 , srv2126511672:87 , srv288170821:87 , srv420554322:87 , srv511221442:87 , srv694730216:87 , srv798380484:87 , srv885195855:87 , srv1086022864:88 , srv380368208:88 , srv581545308:88 , srv757828531:88 , srv852360074:88 , srv1023792292:89 , srv1066459317:89 , srv1459931545:89 , srv1472013188:89 , srv1659873288:89 , srv1758448226:89 , srv1885252248:89 , srv2026853474:89 , srv2093519387:89 , srv426920810:89 , srv52766823:89 , srv609069895:89 , srv665773888:89 , srv1234295313:90 , srv135186097:90 , srv1394722626:90 , srv878370535:90 , srv1071964243:91 , srv133334182:91 , srv1629009354:91 , srv1956171458:91 , srv497083787:91 , srv127939995:92 , srv1060603846:94 , srv418306223:96 , srv25557100:432 }
2023-04-23 02:55:46,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:46,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv798380484=93, srv1175929407=10, srv366675842=66, srv1219323967=12, srv363863194=65, srv1197322044=11, srv135186097=24, srv850412696=94, srv1325591250=21, srv1023792292=2, srv1060603846=4, srv297695370=64, srv1758448226=46, srv188924944=52, srv426920810=70, srv731460424=91, srv497083787=73, srv885195855=98, srv1472013188=34, srv173429722=43, srv1285707754=19, srv514511688=75, srv1762183668=47, srv2107369397=58, srv133334182=22, srv1229358843=13, srv1629009354=38, srv1454873234=32, srv1821442856=49, srv458644154=72, srv2134695562=61, srv1002946250=0, srv1853829030=50, srv1568292427=36, srv159426889=37, srv1659873288=39, srv672289552=84, srv1660505264=40, srv418306223=68, srv1689128018=42, srv1086022864=7, srv1427154718=30, srv420554322=69, srv724528861=89, srv430439604=71, srv1359889875=26, srv2093519387=57, srv1885252248=51, srv1368419522=27, srv546688633=77, srv1686364927=41, srv1793592523=48, srv2126511672=60, srv609069895=80, srv878370535=96, srv711285756=87, srv2026853474=56, srv728709139=90, srv1741967321=45, srv1247369979=15, srv1338367680=23, srv852360074=95, srv1040088768=3, srv1450074241=31, srv511221442=74, srv677914475=85, srv201202780=55, srv195357215=53, srv1250733788=16, srv1319360500=20, srv665773888=83, srv89477090=99, srv658499282=82, srv1394722626=28, srv288170821=63, srv1066459317=5, srv127939995=18, srv612621212=81, srv1071964243=6, srv52766823=76, srv1134830110=8, srv1738742984=44, srv380368208=67, srv880346403=97, srv151089613=35, srv1459931545=33, srv2109679890=59, srv25557100=62, srv757828531=92, srv589232211=79, srv1278229023=17, srv694730216=86, srv1014030431=1, srv1424716894=29, srv1355442980=25, srv718754493=88, srv1956171458=54, srv1234295313=14, srv1146320808=9, srv581545308=78} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:46,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:46,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:46,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:46,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:46,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:46,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:46,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 02:55:46,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 02:55:46,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 02:55:46,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 02:55:46,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 02:55:46,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 02:55:46,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 02:55:46,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 02:55:46,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 02:55:46,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 02:55:46,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 02:55:46,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 02:55:46,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 02:55:46,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 02:55:46,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 02:55:46,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 02:55:47,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038498116845159046, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05971875724033418, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038498116845159046, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:47,127 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 02:55:47,127 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1777561739856045, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038498116845159046, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05971875724033418, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038498116845159046, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 02:55:47,529 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 558 ms to try 5000 different iterations.  Found a solution that moves 2594 regions; Going from a computed imbalance of 0.1777561739856045 to a new imbalance of 0.017266173190899237. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01980828426950348); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03885279710488263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2882222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01980828426950348); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:47,547 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv418306223:78 , srv1060603846:79 , srv497083787:80 , srv1023792292:81 , srv127939995:81 , srv1394722626:81 , srv1459931545:81 , srv52766823:81 , srv878370535:81 , srv1629009354:82 , srv1956171458:82 , srv609069895:82 , srv1014030431:83 , srv1071964243:83 , srv1086022864:83 , srv133334182:83 , srv135186097:83 , srv1472013188:83 , srv1659873288:83 , srv1885252248:83 , srv2026853474:83 , srv380368208:83 , srv665773888:83 , srv1066459317:84 , srv1175929407:84 , srv1234295313:84 , srv159426889:84 , srv1758448226:84 , srv2093519387:84 , srv426920810:84 , srv511221442:84 , srv694730216:84 , srv724528861:84 , srv757828531:84 , srv798380484:84 , srv852360074:84 , srv1040088768:85 , srv1134830110:85 , srv1247369979:85 , srv1355442980:85 , srv1660505264:85 , srv1762183668:85 , srv288170821:85 , srv297695370:85 , srv612621212:85 , srv885195855:85 , srv1219323967:86 , srv1250733788:86 , srv1359889875:86 , srv1689128018:86 , srv420554322:86 , srv514511688:86 , srv589232211:86 , srv672289552:86 , srv677914475:86 , srv711285756:86 , srv1319360500:87 , srv1325591250:87 , srv1738742984:87 , srv1793592523:87 , srv1853829030:87 , srv188924944:87 , srv195357215:87 , srv201202780:87 , srv2109679890:87 , srv366675842:87 , srv658499282:87 , srv850412696:87 , srv880346403:87 , srv1197322044:88 , srv1278229023:88 , srv1285707754:88 , srv1368419522:88 , srv1424716894:88 , srv151089613:88 , srv1686364927:88 , srv173429722:88 , srv2107369397:88 , srv2126511672:88 , srv430439604:88 , srv581545308:88 , srv728709139:88 , srv731460424:88 , srv89477090:88 , srv1146320808:89 , srv1338367680:89 , srv1427154718:89 , srv1454873234:89 , srv1568292427:89 , srv363863194:89 , srv718754493:89 , srv1002946250:90 , srv1229358843:90 , srv1450074241:90 , srv2134695562:90 , srv458644154:90 , srv546688633:90 , srv1741967321:92 , srv1821442856:92 , srv25557100:518 }
2023-04-23 02:55:47,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:47,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv798380484=93, srv1175929407=10, srv366675842=66, srv1219323967=12, srv363863194=65, srv1197322044=11, srv135186097=24, srv850412696=94, srv1325591250=21, srv1023792292=2, srv1060603846=4, srv297695370=64, srv1758448226=46, srv188924944=52, srv426920810=70, srv731460424=91, srv497083787=73, srv885195855=98, srv1472013188=34, srv173429722=43, srv1285707754=19, srv514511688=75, srv1762183668=47, srv2107369397=58, srv133334182=22, srv1229358843=13, srv1629009354=38, srv1454873234=32, srv1821442856=49, srv458644154=72, srv2134695562=61, srv1002946250=0, srv1853829030=50, srv1568292427=36, srv159426889=37, srv1659873288=39, srv672289552=84, srv1660505264=40, srv418306223=68, srv1689128018=42, srv1086022864=7, srv1427154718=30, srv420554322=69, srv724528861=89, srv430439604=71, srv1359889875=26, srv2093519387=57, srv1885252248=51, srv1368419522=27, srv546688633=77, srv1686364927=41, srv1793592523=48, srv2126511672=60, srv609069895=80, srv878370535=96, srv711285756=87, srv2026853474=56, srv728709139=90, srv1741967321=45, srv1247369979=15, srv1338367680=23, srv852360074=95, srv1040088768=3, srv1450074241=31, srv511221442=74, srv677914475=85, srv201202780=55, srv195357215=53, srv1250733788=16, srv1319360500=20, srv665773888=83, srv89477090=99, srv658499282=82, srv1394722626=28, srv288170821=63, srv1066459317=5, srv127939995=18, srv612621212=81, srv1071964243=6, srv52766823=76, srv1134830110=8, srv1738742984=44, srv380368208=67, srv880346403=97, srv151089613=35, srv1459931545=33, srv2109679890=59, srv25557100=62, srv757828531=92, srv589232211=79, srv1278229023=17, srv694730216=86, srv1014030431=1, srv1424716894=29, srv1355442980=25, srv718754493=88, srv1956171458=54, srv1234295313=14, srv1146320808=9, srv581545308=78} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:47,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:47,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:47,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:47,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:47,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 02:55:47,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 02:55:47,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 02:55:47,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 02:55:47,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 02:55:47,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 02:55:47,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 02:55:47,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 02:55:47,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 02:55:47,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01980828426950348); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03885279710488263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01980828426950348); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:47,763 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 02:55:47,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01724800961468134, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01980828426950348); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03885279710488263, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01980828426950348); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 02:55:48,195 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 583 ms to try 5000 different iterations.  Found a solution that moves 1033 regions; Going from a computed imbalance of 0.01724800961468134 to a new imbalance of 6.643043057132737E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012526067096353783); MoveCostFunction : (multiplier=7.0, imbalance=0.11477777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:48,205 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2126511672:81 , srv1234295313:82 , srv151089613:82 , srv581545308:82 , srv1002946250:83 , srv1197322044:83 , srv1285707754:83 , srv133334182:83 , srv1427154718:83 , srv1741967321:83 , srv1853829030:83 , srv420554322:83 , srv458644154:83 , srv89477090:83 , srv1071964243:84 , srv1319360500:84 , srv1359889875:84 , srv1424716894:84 , srv1629009354:84 , srv1686364927:84 , srv1689128018:84 , srv1758448226:84 , srv1793592523:84 , srv188924944:84 , srv2134695562:84 , srv418306223:84 , srv546688633:84 , srv718754493:84 , srv1060603846:85 , srv1066459317:85 , srv1134830110:85 , srv1146320808:85 , srv1229358843:85 , srv1247369979:85 , srv1338367680:85 , srv1450074241:85 , srv1454873234:85 , srv1472013188:85 , srv1659873288:85 , srv2026853474:85 , srv2093519387:85 , srv2109679890:85 , srv363863194:85 , srv426920810:85 , srv430439604:85 , srv609069895:85 , srv658499282:85 , srv665773888:85 , srv677914475:85 , srv711285756:85 , srv728709139:85 , srv731460424:85 , srv757828531:85 , srv880346403:85 , srv885195855:85 , srv1086022864:86 , srv127939995:86 , srv135186097:86 , srv1355442980:86 , srv1368419522:86 , srv1568292427:86 , srv173429722:86 , srv1762183668:86 , srv1821442856:86 , srv1885252248:86 , srv195357215:86 , srv1956171458:86 , srv201202780:86 , srv2107369397:86 , srv288170821:86 , srv366675842:86 , srv514511688:86 , srv589232211:86 , srv672289552:86 , srv694730216:86 , srv850412696:86 , srv852360074:86 , srv878370535:86 , srv1023792292:87 , srv1040088768:87 , srv1219323967:87 , srv1250733788:87 , srv1278229023:87 , srv1325591250:87 , srv1394722626:87 , srv1459931545:87 , srv159426889:87 , srv1660505264:87 , srv1738742984:87 , srv297695370:87 , srv380368208:87 , srv497083787:87 , srv511221442:87 , srv612621212:87 , srv724528861:87 , srv798380484:87 , srv1014030431:88 , srv1175929407:88 , srv52766823:88 , srv25557100:564 }
2023-04-23 02:55:48,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:48,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv798380484=93, srv1175929407=10, srv366675842=66, srv1219323967=12, srv363863194=65, srv1197322044=11, srv135186097=24, srv850412696=94, srv1325591250=21, srv1023792292=2, srv1060603846=4, srv297695370=64, srv1758448226=46, srv188924944=52, srv426920810=70, srv731460424=91, srv497083787=73, srv885195855=98, srv1472013188=34, srv173429722=43, srv1285707754=19, srv514511688=75, srv1762183668=47, srv2107369397=58, srv133334182=22, srv1229358843=13, srv1629009354=38, srv1454873234=32, srv1821442856=49, srv458644154=72, srv2134695562=61, srv1002946250=0, srv1853829030=50, srv1568292427=36, srv159426889=37, srv1659873288=39, srv672289552=84, srv1660505264=40, srv418306223=68, srv1689128018=42, srv1086022864=7, srv1427154718=30, srv420554322=69, srv724528861=89, srv430439604=71, srv1359889875=26, srv2093519387=57, srv1885252248=51, srv1368419522=27, srv546688633=77, srv1686364927=41, srv1793592523=48, srv2126511672=60, srv609069895=80, srv878370535=96, srv711285756=87, srv2026853474=56, srv728709139=90, srv1741967321=45, srv1247369979=15, srv1338367680=23, srv852360074=95, srv1040088768=3, srv1450074241=31, srv511221442=74, srv677914475=85, srv201202780=55, srv195357215=53, srv1250733788=16, srv1319360500=20, srv665773888=83, srv89477090=99, srv658499282=82, srv1394722626=28, srv288170821=63, srv1066459317=5, srv127939995=18, srv612621212=81, srv1071964243=6, srv52766823=76, srv1134830110=8, srv1738742984=44, srv380368208=67, srv880346403=97, srv151089613=35, srv1459931545=33, srv2109679890=59, srv25557100=62, srv757828531=92, srv589232211=79, srv1278229023=17, srv694730216=86, srv1014030431=1, srv1424716894=29, srv1355442980=25, srv718754493=88, srv1956171458=54, srv1234295313=14, srv1146320808=9, srv581545308=78} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:48,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:48,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:48,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:48,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 02:55:48,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 02:55:48,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 02:55:48,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 02:55:48,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 02:55:48,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 02:55:48,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 02:55:48,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 02:55:48,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 02:55:48,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 02:55:48,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 02:55:48,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 02:55:48,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 02:55:48,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 02:55:48,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 02:55:48,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 02:55:48,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 02:55:48,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 02:55:48,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 02:55:48,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 02:55:48,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 02:55:48,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 02:55:48,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 02:55:48,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 02:55:48,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 02:55:48,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 02:55:48,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 02:55:48,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 02:55:48,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 02:55:48,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 02:55:48,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 02:55:48,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 02:55:48,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 02:55:48,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 02:55:48,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 02:55:48,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 02:55:48,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 02:55:48,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 02:55:48,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 02:55:48,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012526067096353783); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:48,429 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-23 02:55:48,432 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.570710851145499E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012526067096353783); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-23 02:55:48,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 557 ms to try 5000 different iterations.  Found a solution that moves 270 regions; Going from a computed imbalance of 6.570710851145499E-4 to a new imbalance of 1.8932082532752226E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.03, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:48,851 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv363863194:82 , srv127939995:83 , srv135186097:83 , srv1368419522:83 , srv1821442856:83 , srv1956171458:83 , srv1002946250:84 , srv1014030431:84 , srv1060603846:84 , srv1175929407:84 , srv1197322044:84 , srv1219323967:84 , srv1250733788:84 , srv1278229023:84 , srv1338367680:84 , srv1359889875:84 , srv1450074241:84 , srv1568292427:84 , srv1660505264:84 , srv1689128018:84 , srv1738742984:84 , srv1741967321:84 , srv1853829030:84 , srv1885252248:84 , srv195357215:84 , srv2093519387:84 , srv297695370:84 , srv366675842:84 , srv380368208:84 , srv418306223:84 , srv426920810:84 , srv497083787:84 , srv511221442:84 , srv514511688:84 , srv52766823:84 , srv581545308:84 , srv589232211:84 , srv672289552:84 , srv724528861:84 , srv798380484:84 , srv850412696:84 , srv852360074:84 , srv1023792292:85 , srv1040088768:85 , srv1066459317:85 , srv1071964243:85 , srv1086022864:85 , srv1134830110:85 , srv1146320808:85 , srv1229358843:85 , srv1234295313:85 , srv1247369979:85 , srv1319360500:85 , srv1325591250:85 , srv133334182:85 , srv1355442980:85 , srv1394722626:85 , srv1424716894:85 , srv1427154718:85 , srv1454873234:85 , srv1459931545:85 , srv1472013188:85 , srv159426889:85 , srv1629009354:85 , srv1659873288:85 , srv1686364927:85 , srv173429722:85 , srv1758448226:85 , srv1762183668:85 , srv188924944:85 , srv201202780:85 , srv2026853474:85 , srv2107369397:85 , srv2109679890:85 , srv2134695562:85 , srv288170821:85 , srv430439604:85 , srv458644154:85 , srv546688633:85 , srv612621212:85 , srv658499282:85 , srv665773888:85 , srv677914475:85 , srv694730216:85 , srv711285756:85 , srv718754493:85 , srv728709139:85 , srv731460424:85 , srv757828531:85 , srv878370535:85 , srv880346403:85 , srv885195855:85 , srv89477090:85 , srv1285707754:86 , srv151089613:86 , srv1793592523:86 , srv2126511672:86 , srv420554322:86 , srv609069895:86 , srv25557100:628 }
2023-04-23 02:55:48,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:48,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv798380484=93, srv1175929407=10, srv366675842=66, srv1219323967=12, srv363863194=65, srv1197322044=11, srv135186097=24, srv850412696=94, srv1325591250=21, srv1023792292=2, srv1060603846=4, srv297695370=64, srv1758448226=46, srv188924944=52, srv426920810=70, srv731460424=91, srv497083787=73, srv885195855=98, srv1472013188=34, srv173429722=43, srv1285707754=19, srv514511688=75, srv1762183668=47, srv2107369397=58, srv133334182=22, srv1229358843=13, srv1629009354=38, srv1454873234=32, srv1821442856=49, srv458644154=72, srv2134695562=61, srv1002946250=0, srv1853829030=50, srv1568292427=36, srv159426889=37, srv1659873288=39, srv672289552=84, srv1660505264=40, srv418306223=68, srv1689128018=42, srv1086022864=7, srv1427154718=30, srv420554322=69, srv724528861=89, srv430439604=71, srv1359889875=26, srv2093519387=57, srv1885252248=51, srv1368419522=27, srv546688633=77, srv1686364927=41, srv1793592523=48, srv2126511672=60, srv609069895=80, srv878370535=96, srv711285756=87, srv2026853474=56, srv728709139=90, srv1741967321=45, srv1247369979=15, srv1338367680=23, srv852360074=95, srv1040088768=3, srv1450074241=31, srv511221442=74, srv677914475=85, srv201202780=55, srv195357215=53, srv1250733788=16, srv1319360500=20, srv665773888=83, srv89477090=99, srv658499282=82, srv1394722626=28, srv288170821=63, srv1066459317=5, srv127939995=18, srv612621212=81, srv1071964243=6, srv52766823=76, srv1134830110=8, srv1738742984=44, srv380368208=67, srv880346403=97, srv151089613=35, srv1459931545=33, srv2109679890=59, srv25557100=62, srv757828531=92, srv589232211=79, srv1278229023=17, srv694730216=86, srv1014030431=1, srv1424716894=29, srv1355442980=25, srv718754493=88, srv1956171458=54, srv1234295313=14, srv1146320808=9, srv581545308=78} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:48,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:48,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:48,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:48,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-23 02:55:48,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-23 02:55:48,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-23 02:55:48,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-23 02:55:48,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-23 02:55:48,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-23 02:55:48,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-23 02:55:48,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-23 02:55:48,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-23 02:55:48,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-23 02:55:48,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-23 02:55:48,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-23 02:55:48,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-23 02:55:49,030 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7041502124566916E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:49,031 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv363863194:82 , srv127939995:83 , srv135186097:83 , srv1368419522:83 , srv1821442856:83 , srv1956171458:83 , srv1002946250:84 , srv1014030431:84 , srv1060603846:84 , srv1175929407:84 , srv1197322044:84 , srv1219323967:84 , srv1250733788:84 , srv1278229023:84 , srv1338367680:84 , srv1359889875:84 , srv1450074241:84 , srv1568292427:84 , srv1660505264:84 , srv1689128018:84 , srv1738742984:84 , srv1741967321:84 , srv1853829030:84 , srv1885252248:84 , srv195357215:84 , srv2093519387:84 , srv297695370:84 , srv366675842:84 , srv380368208:84 , srv418306223:84 , srv426920810:84 , srv497083787:84 , srv511221442:84 , srv514511688:84 , srv52766823:84 , srv581545308:84 , srv589232211:84 , srv672289552:84 , srv724528861:84 , srv798380484:84 , srv850412696:84 , srv852360074:84 , srv1023792292:85 , srv1040088768:85 , srv1066459317:85 , srv1071964243:85 , srv1086022864:85 , srv1134830110:85 , srv1146320808:85 , srv1229358843:85 , srv1234295313:85 , srv1247369979:85 , srv1319360500:85 , srv1325591250:85 , srv133334182:85 , srv1355442980:85 , srv1394722626:85 , srv1424716894:85 , srv1427154718:85 , srv1454873234:85 , srv1459931545:85 , srv1472013188:85 , srv159426889:85 , srv1629009354:85 , srv1659873288:85 , srv1686364927:85 , srv173429722:85 , srv1758448226:85 , srv1762183668:85 , srv188924944:85 , srv201202780:85 , srv2026853474:85 , srv2107369397:85 , srv2109679890:85 , srv2134695562:85 , srv288170821:85 , srv430439604:85 , srv458644154:85 , srv546688633:85 , srv612621212:85 , srv658499282:85 , srv665773888:85 , srv677914475:85 , srv694730216:85 , srv711285756:85 , srv718754493:85 , srv728709139:85 , srv731460424:85 , srv757828531:85 , srv878370535:85 , srv880346403:85 , srv885195855:85 , srv89477090:85 , srv1285707754:86 , srv151089613:86 , srv1793592523:86 , srv2126511672:86 , srv420554322:86 , srv609069895:86 , srv25557100:628 }
2023-04-23 02:55:49,072 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=412 (was 369) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=12707 (was 13202)
2023-04-23 02:55:49,083 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=412, ProcessCount=170, AvailableMemoryMB=12707
2023-04-23 02:55:49,083 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-23 02:55:49,084 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-23 02:55:49,098 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1051051441:84 , srv1063002690:84 , srv1121026720:84 , srv1338890933:84 , srv1347306544:84 , srv1469334877:84 , srv1531841212:84 , srv1610519434:84 , srv1629020767:84 , srv1762793201:84 , srv1773762118:84 , srv1775612105:84 , srv1866322159:84 , srv1876719492:84 , srv2025311004:84 , srv2055084150:84 , srv2136958552:84 , srv29375790:84 , srv398075815:84 , srv405375880:84 , srv499810197:84 , srv536812978:84 , srv543123364:84 , srv567382677:84 , srv580118065:84 , srv707836686:84 , srv866662892:84 , srv911534985:84 , srv97101344:84 , srv2043533322:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-23 02:55:49,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:55:49,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2136958552=17, srv1051051441=0, srv1876719492=13, srv536812978=22, srv405375880=20, srv1121026720=2, srv2055084150=16, srv911534985=28, srv398075815=19, srv1775612105=11, srv543123364=23, srv1063002690=1, srv1469334877=5, srv1762793201=9, srv580118065=25, srv1629020767=8, srv1610519434=7, srv499810197=21, srv1347306544=4, srv1773762118=10, srv2043533322=15, srv1866322159=12, srv97101344=29, srv707836686=26, srv1531841212=6, srv29375790=18, srv1338890933=3, srv567382677=24, srv2025311004=14, srv866662892=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:55:49,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:55:49,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:55:49,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-23 02:55:49,138 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1428257313666, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:55:49,138 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912643082480126, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1428257313666, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-23 02:57:49,120 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5679166 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912643082480126 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:57:49,126 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1051051441:90 , srv1063002690:90 , srv1121026720:90 , srv1338890933:90 , srv1347306544:90 , srv1469334877:90 , srv1531841212:90 , srv1610519434:90 , srv1629020767:90 , srv1762793201:90 , srv1773762118:90 , srv1775612105:90 , srv1866322159:90 , srv1876719492:90 , srv2025311004:90 , srv2043533322:90 , srv2055084150:90 , srv2136958552:90 , srv29375790:90 , srv398075815:90 , srv405375880:90 , srv499810197:90 , srv536812978:90 , srv543123364:90 , srv567382677:90 , srv580118065:90 , srv707836686:90 , srv866662892:90 , srv911534985:90 , srv97101344:90 }
2023-04-23 02:57:49,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-23 02:57:49,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2136958552=17, srv1051051441=0, srv1876719492=13, srv536812978=22, srv405375880=20, srv1121026720=2, srv2055084150=16, srv911534985=28, srv398075815=19, srv1775612105=11, srv543123364=23, srv1063002690=1, srv1469334877=5, srv1762793201=9, srv580118065=25, srv1629020767=8, srv1610519434=7, srv499810197=21, srv1347306544=4, srv1773762118=10, srv2043533322=15, srv1866322159=12, srv97101344=29, srv707836686=26, srv1531841212=6, srv29375790=18, srv1338890933=3, srv567382677=24, srv2025311004=14, srv866662892=27} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-23 02:57:49,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-23 02:57:49,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-23 02:57:49,145 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-23 02:57:49,145 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1051051441:90 , srv1063002690:90 , srv1121026720:90 , srv1338890933:90 , srv1347306544:90 , srv1469334877:90 , srv1531841212:90 , srv1610519434:90 , srv1629020767:90 , srv1762793201:90 , srv1773762118:90 , srv1775612105:90 , srv1866322159:90 , srv1876719492:90 , srv2025311004:90 , srv2043533322:90 , srv2055084150:90 , srv2136958552:90 , srv29375790:90 , srv398075815:90 , srv405375880:90 , srv499810197:90 , srv536812978:90 , srv543123364:90 , srv567382677:90 , srv580118065:90 , srv707836686:90 , srv866662892:90 , srv911534985:90 , srv97101344:90 }
2023-04-23 02:57:49,157 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=282 (was 412), ProcessCount=169 (was 170), AvailableMemoryMB=12759 (was 12707) - AvailableMemoryMB LEAK? -