2023-03-26 11:55:59,156 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-26 11:55:59,472 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-26 11:55:59,574 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-26 11:55:59,630 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-26 11:55:59,637 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-26 11:55:59,649 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=361, ProcessCount=169, AvailableMemoryMB=9549
2023-03-26 11:55:59,651 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-26 11:55:59,651 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-26 11:55:59,781 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv10124337:84 , srv1019593531:84 , srv1028540296:84 , srv1050179271:84 , srv1086295409:84 , srv1095746124:84 , srv1123380178:84 , srv1123436504:84 , srv1144917673:84 , srv1181527761:84 , srv1196524605:84 , srv1204840180:84 , srv1217292493:84 , srv1276569525:84 , srv1323607107:84 , srv1329679715:84 , srv1337624465:84 , srv1350963048:84 , srv135537008:84 , srv1359031748:84 , srv141097326:84 , srv143812960:84 , srv1441845358:84 , srv1452815541:84 , srv1486365445:84 , srv1517813023:84 , srv1534703178:84 , srv1559397214:84 , srv1564699328:84 , srv1577884768:84 , srv1586709337:84 , srv1590725487:84 , srv1595074490:84 , srv1705639404:84 , srv1733408656:84 , srv174928315:84 , srv1807044098:84 , srv1831503129:84 , srv1850582312:84 , srv1850974246:84 , srv1874263445:84 , srv1902126797:84 , srv1920810693:84 , srv1930413468:84 , srv1934471273:84 , srv196111901:84 , srv1962199476:84 , srv1970162107:84 , srv1993369150:84 , srv2004257953:84 , srv2010730358:84 , srv201935819:84 , srv2024838133:84 , srv2037702928:84 , srv209066409:84 , srv2105034058:84 , srv2110930807:84 , srv2111458830:84 , srv2117882384:84 , srv2131915391:84 , srv2137836246:84 , srv230483651:84 , srv290192280:84 , srv300897613:84 , srv316633381:84 , srv331987249:84 , srv344639101:84 , srv349646824:84 , srv400761592:84 , srv404596528:84 , srv418129458:84 , srv445919309:84 , srv446002750:84 , srv454442920:84 , srv459386037:84 , srv464035570:84 , srv476544108:84 , srv508694834:84 , srv523319703:84 , srv53917021:84 , srv582159799:84 , srv616159868:84 , srv668158909:84 , srv668368317:84 , srv670538537:84 , srv694746665:84 , srv742962169:84 , srv777496296:84 , srv797555924:84 , srv802139301:84 , srv82314379:84 , srv855748724:84 , srv873155076:84 , srv875973947:84 , srv919906228:84 , srv923728238:84 , srv93275082:84 , srv942267995:84 , srv99572466:84 , srv1901142752:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-26 11:55:59,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:55:59,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:55:59,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:55:59,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:55:59,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:55:59,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:55:59,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:55:59,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:55:59,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:55:59,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:55:59,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:55:59,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:55:59,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:55:59,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:55:59,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:00,195 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:00,196 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-26 11:56:00,196 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-26 11:56:00,848 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 917 ms to try 5000 different iterations.  Found a solution that moves 4636 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18728263560606004. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04102988912939347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04947309350465477, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5151111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04102988912939347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:00,869 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1850974246:82 , srv1962199476:82 , srv919906228:82 , srv93275082:82 , srv10124337:83 , srv1095746124:83 , srv1452815541:83 , srv201935819:83 , srv2037702928:83 , srv454442920:83 , srv1204840180:84 , srv1441845358:84 , srv1559397214:84 , srv1930413468:84 , srv2131915391:84 , srv290192280:84 , srv344639101:84 , srv349646824:84 , srv418129458:84 , srv446002750:84 , srv135537008:85 , srv1586709337:85 , srv2010730358:85 , srv2117882384:85 , srv230483651:85 , srv316633381:85 , srv464035570:85 , srv582159799:85 , srv616159868:85 , srv777496296:85 , srv942267995:85 , srv1050179271:86 , srv1144917673:86 , srv1196524605:86 , srv1329679715:86 , srv1337624465:86 , srv1350963048:86 , srv1486365445:86 , srv1534703178:86 , srv1874263445:86 , srv196111901:86 , srv1993369150:86 , srv2024838133:86 , srv2110930807:86 , srv331987249:86 , srv445919309:86 , srv523319703:86 , srv668368317:86 , srv694746665:86 , srv802139301:86 , srv875973947:86 , srv923728238:86 , srv1028540296:87 , srv1123436504:87 , srv1181527761:87 , srv1323607107:87 , srv1359031748:87 , srv141097326:87 , srv1577884768:87 , srv1595074490:87 , srv1705639404:87 , srv174928315:87 , srv1850582312:87 , srv1902126797:87 , srv1920810693:87 , srv1970162107:87 , srv209066409:87 , srv2105034058:87 , srv404596528:87 , srv508694834:87 , srv53917021:87 , srv670538537:87 , srv82314379:87 , srv99572466:87 , srv1123380178:88 , srv143812960:88 , srv1564699328:88 , srv1733408656:88 , srv1831503129:88 , srv2111458830:88 , srv400761592:88 , srv459386037:88 , srv668158909:88 , srv742962169:88 , srv1019593531:89 , srv1217292493:89 , srv1517813023:89 , srv1590725487:89 , srv2004257953:89 , srv2137836246:89 , srv300897613:89 , srv476544108:89 , srv873155076:89 , srv1086295409:90 , srv1807044098:90 , srv797555924:90 , srv1276569525:91 , srv1934471273:91 , srv855748724:91 , srv1901142752:455 }
2023-03-26 11:56:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:00,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:00,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:56:00,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:56:00,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:56:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:56:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:56:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:56:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:01,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04102988912939347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04947309350465477, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04102988912939347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:01,171 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-26 11:56:01,171 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18725017364023652, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04102988912939347, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04947309350465477, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04102988912939347, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1825, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25025, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-26 11:56:01,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 535 ms to try 5000 different iterations.  Found a solution that moves 2700 regions; Going from a computed imbalance of 0.18725017364023652 to a new imbalance of 0.01491549542866685. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021490187484627825); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028009137518170468, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.3, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021490187484627825); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:01,473 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1276569525:80 , srv1934471273:80 , srv873155076:80 , srv1590725487:81 , srv1019593531:82 , srv1807044098:82 , srv2004257953:82 , srv2111458830:82 , srv300897613:82 , srv476544108:82 , srv742962169:82 , srv797555924:82 , srv855748724:82 , srv1217292493:83 , srv143812960:83 , srv1564699328:83 , srv459386037:83 , srv670538537:83 , srv1337624465:84 , srv1705639404:84 , srv1831503129:84 , srv1970162107:84 , srv2137836246:84 , srv331987249:84 , srv404596528:84 , srv82314379:84 , srv1028540296:85 , srv1086295409:85 , srv1181527761:85 , srv1350963048:85 , srv1534703178:85 , srv1577884768:85 , srv1586709337:85 , srv1733408656:85 , srv1902126797:85 , srv1920810693:85 , srv196111901:85 , srv209066409:85 , srv400761592:85 , srv53917021:85 , srv668158909:85 , srv668368317:85 , srv875973947:85 , srv1144917673:86 , srv1323607107:86 , srv1359031748:86 , srv141097326:86 , srv1486365445:86 , srv1517813023:86 , srv1595074490:86 , srv174928315:86 , srv1850582312:86 , srv1993369150:86 , srv2024838133:86 , srv2105034058:86 , srv2117882384:86 , srv344639101:86 , srv464035570:86 , srv508694834:86 , srv616159868:86 , srv694746665:86 , srv802139301:86 , srv923728238:86 , srv99572466:86 , srv1123380178:87 , srv1196524605:87 , srv1441845358:87 , srv1874263445:87 , srv1930413468:87 , srv2131915391:87 , srv230483651:87 , srv418129458:87 , srv445919309:87 , srv446002750:87 , srv523319703:87 , srv777496296:87 , srv1095746124:88 , srv1123436504:88 , srv135537008:88 , srv2010730358:88 , srv201935819:88 , srv2110930807:88 , srv316633381:88 , srv349646824:88 , srv582159799:88 , srv942267995:88 , srv1204840180:89 , srv2037702928:89 , srv290192280:89 , srv93275082:89 , srv10124337:90 , srv1050179271:90 , srv1329679715:90 , srv1850974246:90 , srv1962199476:90 , srv919906228:90 , srv1452815541:91 , srv1559397214:91 , srv454442920:91 , srv1901142752:510 }
2023-03-26 11:56:01,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:01,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:01,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:01,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:56:01,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:56:01,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:56:01,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:56:01,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:56:01,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:01,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021490187484627825); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028009137518170468, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021490187484627825); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:01,643 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-26 11:56:01,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014896589624584997, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021490187484627825); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.028009137518170468, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021490187484627825); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04708333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-26 11:56:01,903 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 373 ms to try 5000 different iterations.  Found a solution that moves 967 regions; Going from a computed imbalance of 0.014896589624584997 to a new imbalance of 0.0010127236229171498. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.10744444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:01,905 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1050179271:81 , srv1329679715:81 , srv1086295409:82 , srv1123436504:82 , srv1123380178:83 , srv1204840180:83 , srv1452815541:83 , srv1517813023:83 , srv1559397214:83 , srv454442920:83 , srv10124337:84 , srv1323607107:84 , srv1733408656:84 , srv174928315:84 , srv1962199476:84 , srv2105034058:84 , srv2110930807:84 , srv2137836246:84 , srv290192280:84 , srv316633381:84 , srv400761592:84 , srv445919309:84 , srv582159799:84 , srv668158909:84 , srv942267995:84 , srv1181527761:85 , srv1196524605:85 , srv135537008:85 , srv1359031748:85 , srv141097326:85 , srv1486365445:85 , srv1577884768:85 , srv1831503129:85 , srv1850582312:85 , srv1850974246:85 , srv1874263445:85 , srv2010730358:85 , srv2024838133:85 , srv209066409:85 , srv349646824:85 , srv523319703:85 , srv694746665:85 , srv855748724:85 , srv919906228:85 , srv99572466:85 , srv1019593531:86 , srv1028540296:86 , srv1144917673:86 , srv1217292493:86 , srv1276569525:86 , srv1350963048:86 , srv143812960:86 , srv1441845358:86 , srv1534703178:86 , srv1590725487:86 , srv1595074490:86 , srv1705639404:86 , srv1807044098:86 , srv1902126797:86 , srv1920810693:86 , srv1930413468:86 , srv196111901:86 , srv1970162107:86 , srv1993369150:86 , srv2004257953:86 , srv2037702928:86 , srv2111458830:86 , srv2131915391:86 , srv230483651:86 , srv300897613:86 , srv404596528:86 , srv459386037:86 , srv464035570:86 , srv476544108:86 , srv508694834:86 , srv53917021:86 , srv616159868:86 , srv668368317:86 , srv742962169:86 , srv777496296:86 , srv797555924:86 , srv802139301:86 , srv82314379:86 , srv923728238:86 , srv1095746124:87 , srv1337624465:87 , srv1564699328:87 , srv1934471273:87 , srv201935819:87 , srv344639101:87 , srv418129458:87 , srv446002750:87 , srv875973947:87 , srv1586709337:88 , srv2117882384:88 , srv670538537:88 , srv873155076:88 , srv93275082:88 , srv331987249:90 , srv1901142752:549 }
2023-03-26 11:56:01,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:01,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:56:01,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:56:01,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:56:01,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:56:01,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:56:01,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:02,030 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:02,031 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-26 11:56:02,031 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0010059525441959824, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063328312164356885); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00670025210172808); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063328312164356885); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-26 11:56:02,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 420 ms to try 5000 different iterations.  Found a solution that moves 231 regions; Going from a computed imbalance of 0.0010059525441959824 to a new imbalance of 4.95228630919196E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.025666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:02,358 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1595074490:83 , srv1934471273:83 , srv201935819:83 , srv2110930807:83 , srv2117882384:83 , srv331987249:83 , srv508694834:83 , srv93275082:83 , srv1028540296:84 , srv1095746124:84 , srv1123380178:84 , srv1144917673:84 , srv1196524605:84 , srv135537008:84 , srv1359031748:84 , srv141097326:84 , srv1517813023:84 , srv1559397214:84 , srv1564699328:84 , srv1586709337:84 , srv1807044098:84 , srv1850582312:84 , srv1874263445:84 , srv1920810693:84 , srv1993369150:84 , srv2010730358:84 , srv2024838133:84 , srv2037702928:84 , srv418129458:84 , srv446002750:84 , srv459386037:84 , srv476544108:84 , srv523319703:84 , srv53917021:84 , srv670538537:84 , srv777496296:84 , srv797555924:84 , srv802139301:84 , srv855748724:84 , srv875973947:84 , srv923728238:84 , srv99572466:84 , srv10124337:85 , srv1019593531:85 , srv1050179271:85 , srv1086295409:85 , srv1123436504:85 , srv1181527761:85 , srv1217292493:85 , srv1276569525:85 , srv1323607107:85 , srv1329679715:85 , srv1337624465:85 , srv1350963048:85 , srv143812960:85 , srv1441845358:85 , srv1452815541:85 , srv1486365445:85 , srv1534703178:85 , srv1577884768:85 , srv1705639404:85 , srv1733408656:85 , srv174928315:85 , srv1831503129:85 , srv1850974246:85 , srv1902126797:85 , srv1930413468:85 , srv196111901:85 , srv1970162107:85 , srv2004257953:85 , srv209066409:85 , srv2105034058:85 , srv2137836246:85 , srv230483651:85 , srv290192280:85 , srv300897613:85 , srv316633381:85 , srv344639101:85 , srv349646824:85 , srv400761592:85 , srv445919309:85 , srv454442920:85 , srv464035570:85 , srv582159799:85 , srv616159868:85 , srv668158909:85 , srv668368317:85 , srv694746665:85 , srv82314379:85 , srv873155076:85 , srv919906228:85 , srv942267995:85 , srv1204840180:86 , srv1590725487:86 , srv1962199476:86 , srv2111458830:86 , srv2131915391:86 , srv404596528:86 , srv742962169:86 , srv1901142752:628 }
2023-03-26 11:56:02,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:02,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:02,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:02,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:02,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:56:02,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:56:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:56:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:56:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:56:02,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:56:02,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:56:02,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:02,490 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:02,490 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-26 11:56:02,491 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.790536652047217E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-26 11:56:02,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 304 ms to try 5000 different iterations.  Found a solution that moves 26 regions; Going from a computed imbalance of 4.790536652047217E-5 to a new imbalance of 7.569813391471145E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0028888888888888888); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:02,698 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1217292493:83 , srv1902126797:83 , srv2131915391:83 , srv230483651:83 , srv331987249:83 , srv404596528:83 , srv10124337:84 , srv1019593531:84 , srv1028540296:84 , srv1050179271:84 , srv1086295409:84 , srv1095746124:84 , srv1123380178:84 , srv1123436504:84 , srv1144917673:84 , srv1181527761:84 , srv1196524605:84 , srv1204840180:84 , srv1276569525:84 , srv1323607107:84 , srv1329679715:84 , srv1337624465:84 , srv1350963048:84 , srv135537008:84 , srv1359031748:84 , srv141097326:84 , srv143812960:84 , srv1441845358:84 , srv1452815541:84 , srv1486365445:84 , srv1517813023:84 , srv1534703178:84 , srv1559397214:84 , srv1564699328:84 , srv1577884768:84 , srv1586709337:84 , srv1590725487:84 , srv1595074490:84 , srv1705639404:84 , srv1733408656:84 , srv174928315:84 , srv1807044098:84 , srv1831503129:84 , srv1850582312:84 , srv1850974246:84 , srv1874263445:84 , srv1920810693:84 , srv1930413468:84 , srv196111901:84 , srv1962199476:84 , srv1970162107:84 , srv1993369150:84 , srv2004257953:84 , srv2010730358:84 , srv2037702928:84 , srv209066409:84 , srv2105034058:84 , srv2111458830:84 , srv2117882384:84 , srv2137836246:84 , srv290192280:84 , srv300897613:84 , srv316633381:84 , srv344639101:84 , srv349646824:84 , srv400761592:84 , srv418129458:84 , srv445919309:84 , srv446002750:84 , srv454442920:84 , srv464035570:84 , srv508694834:84 , srv523319703:84 , srv53917021:84 , srv582159799:84 , srv616159868:84 , srv668158909:84 , srv668368317:84 , srv670538537:84 , srv694746665:84 , srv742962169:84 , srv777496296:84 , srv797555924:84 , srv802139301:84 , srv82314379:84 , srv855748724:84 , srv873155076:84 , srv875973947:84 , srv919906228:84 , srv923728238:84 , srv93275082:84 , srv942267995:84 , srv99572466:84 , srv1934471273:85 , srv201935819:85 , srv2024838133:85 , srv2110930807:85 , srv459386037:85 , srv476544108:85 , srv1901142752:684 }
2023-03-26 11:56:02,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:02,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1733408656=34, srv404596528=70, srv797555924=89, srv2117882384=59, srv400761592=69, srv141097326=20, srv2024838133=53, srv2137836246=61, srv855748724=92, srv1920810693=43, srv349646824=68, srv668158909=83, srv670538537=85, srv1086295409=4, srv1705639404=33, srv1019593531=1, srv10124337=0, srv464035570=76, srv1807044098=36, srv1850582312=38, srv1276569525=13, srv1586709337=30, srv942267995=98, srv459386037=75, srv1323607107=14, srv2110930807=57, srv668368317=84, srv344639101=67, srv1050179271=3, srv1970162107=48, srv93275082=97, srv1993369150=49, srv445919309=72, srv1217292493=12, srv2111458830=58, srv135537008=18, srv802139301=90, srv290192280=63, srv1486365445=24, srv1204840180=11, srv1517813023=25, srv1196524605=10, srv1181527761=9, srv1559397214=27, srv1359031748=19, srv1901142752=41, srv694746665=86, srv196111901=46, srv2131915391=60, srv1902126797=42, srv82314379=91, srv1441845358=22, srv1962199476=47, srv2010730358=51, srv2037702928=54, srv1452815541=23, srv53917021=80, srv873155076=93, srv300897613=64, srv1874263445=40, srv923728238=96, srv476544108=77, srv143812960=21, srv230483651=62, srv331987249=66, srv454442920=74, srv316633381=65, srv742962169=87, srv1850974246=39, srv1350963048=17, srv1534703178=26, srv523319703=79, srv1123380178=6, srv99572466=99, srv201935819=52, srv2105034058=56, srv1028540296=2, srv1123436504=7, srv1337624465=16, srv777496296=88, srv2004257953=50, srv209066409=55, srv875973947=94, srv1595074490=32, srv1930413468=44, srv1934471273=45, srv446002750=73, srv508694834=78, srv1095746124=5, srv1577884768=29, srv1144917673=8, srv616159868=82, srv174928315=35, srv1590725487=31, srv418129458=71, srv919906228=95, srv1329679715=15, srv582159799=81, srv1564699328=28, srv1831503129=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:02,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:02,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:02,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-26 11:56:02,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-26 11:56:02,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-26 11:56:02,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-26 11:56:02,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-26 11:56:02,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-26 11:56:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-26 11:56:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-26 11:56:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-26 11:56:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-26 11:56:02,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-26 11:56:02,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-26 11:56:02,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-26 11:56:02,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-26 11:56:02,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-26 11:56:02,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-26 11:56:02,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-26 11:56:02,871 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:02,871 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1217292493:83 , srv1902126797:83 , srv2131915391:83 , srv230483651:83 , srv331987249:83 , srv404596528:83 , srv10124337:84 , srv1019593531:84 , srv1028540296:84 , srv1050179271:84 , srv1086295409:84 , srv1095746124:84 , srv1123380178:84 , srv1123436504:84 , srv1144917673:84 , srv1181527761:84 , srv1196524605:84 , srv1204840180:84 , srv1276569525:84 , srv1323607107:84 , srv1329679715:84 , srv1337624465:84 , srv1350963048:84 , srv135537008:84 , srv1359031748:84 , srv141097326:84 , srv143812960:84 , srv1441845358:84 , srv1452815541:84 , srv1486365445:84 , srv1517813023:84 , srv1534703178:84 , srv1559397214:84 , srv1564699328:84 , srv1577884768:84 , srv1586709337:84 , srv1590725487:84 , srv1595074490:84 , srv1705639404:84 , srv1733408656:84 , srv174928315:84 , srv1807044098:84 , srv1831503129:84 , srv1850582312:84 , srv1850974246:84 , srv1874263445:84 , srv1920810693:84 , srv1930413468:84 , srv196111901:84 , srv1962199476:84 , srv1970162107:84 , srv1993369150:84 , srv2004257953:84 , srv2010730358:84 , srv2037702928:84 , srv209066409:84 , srv2105034058:84 , srv2111458830:84 , srv2117882384:84 , srv2137836246:84 , srv290192280:84 , srv300897613:84 , srv316633381:84 , srv344639101:84 , srv349646824:84 , srv400761592:84 , srv418129458:84 , srv445919309:84 , srv446002750:84 , srv454442920:84 , srv464035570:84 , srv508694834:84 , srv523319703:84 , srv53917021:84 , srv582159799:84 , srv616159868:84 , srv668158909:84 , srv668368317:84 , srv670538537:84 , srv694746665:84 , srv742962169:84 , srv777496296:84 , srv797555924:84 , srv802139301:84 , srv82314379:84 , srv855748724:84 , srv873155076:84 , srv875973947:84 , srv919906228:84 , srv923728238:84 , srv93275082:84 , srv942267995:84 , srv99572466:84 , srv1934471273:85 , srv201935819:85 , srv2024838133:85 , srv2110930807:85 , srv459386037:85 , srv476544108:85 , srv1901142752:684 }
2023-03-26 11:56:02,925 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=361 (was 361), ProcessCount=169 (was 169), AvailableMemoryMB=9044 (was 9549)
2023-03-26 11:56:02,939 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=262, MaxFileDescriptor=60000, SystemLoadAverage=361, ProcessCount=169, AvailableMemoryMB=9043
2023-03-26 11:56:02,940 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-26 11:56:02,940 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-26 11:56:02,954 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1086125432:84 , srv1156869425:84 , srv1160056597:84 , srv1239798910:84 , srv1288161943:84 , srv12887284:84 , srv1334931806:84 , srv138351398:84 , srv1503596119:84 , srv1508491888:84 , srv1514141767:84 , srv1547772868:84 , srv1579684215:84 , srv1598113064:84 , srv1673540513:84 , srv1894043367:84 , srv2001695170:84 , srv2051433281:84 , srv383514237:84 , srv420391003:84 , srv45522112:84 , srv478257586:84 , srv521967606:84 , srv586763432:84 , srv674356432:84 , srv734704649:84 , srv813502877:84 , srv814974893:84 , srv967150906:84 , srv1306968728:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-26 11:56:02,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:56:02,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1514141767=11, srv1239798910=3, srv967150906=29, srv2051433281=18, srv734704649=26, srv1547772868=12, srv1156869425=1, srv1160056597=2, srv521967606=23, srv2001695170=17, srv45522112=21, srv586763432=24, srv1334931806=7, srv813502877=27, srv12887284=5, srv674356432=25, srv478257586=22, srv1508491888=10, srv383514237=19, srv420391003=20, srv1306968728=6, srv1579684215=13, srv1894043367=16, srv1598113064=14, srv138351398=8, srv814974893=28, srv1503596119=9, srv1673540513=15, srv1086125432=0, srv1288161943=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:56:02,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:56:02,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:56:02,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-26 11:56:03,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1156759558783365, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:56:03,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912557534424085, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1156759558783365, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-26 11:58:02,986 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6313435 different iterations.  Found a solution that moves 1862 regions; Going from a computed imbalance of 0.9912557534424085 to a new imbalance of 4.346000888939571E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6896296296296296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:58:02,992 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1086125432:90 , srv1156869425:90 , srv1160056597:90 , srv1239798910:90 , srv1288161943:90 , srv12887284:90 , srv1306968728:90 , srv1334931806:90 , srv138351398:90 , srv1503596119:90 , srv1508491888:90 , srv1514141767:90 , srv1547772868:90 , srv1579684215:90 , srv1598113064:90 , srv1673540513:90 , srv1894043367:90 , srv2001695170:90 , srv2051433281:90 , srv383514237:90 , srv420391003:90 , srv45522112:90 , srv478257586:90 , srv521967606:90 , srv586763432:90 , srv674356432:90 , srv734704649:90 , srv813502877:90 , srv814974893:90 , srv967150906:90 }
2023-03-26 11:58:02,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-26 11:58:02,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1514141767=11, srv1239798910=3, srv967150906=29, srv2051433281=18, srv734704649=26, srv1547772868=12, srv1156869425=1, srv1160056597=2, srv521967606=23, srv2001695170=17, srv45522112=21, srv586763432=24, srv1334931806=7, srv813502877=27, srv12887284=5, srv674356432=25, srv478257586=22, srv1508491888=10, srv383514237=19, srv420391003=20, srv1306968728=6, srv1579684215=13, srv1894043367=16, srv1598113064=14, srv138351398=8, srv814974893=28, srv1503596119=9, srv1673540513=15, srv1086125432=0, srv1288161943=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-26 11:58:02,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-26 11:58:02,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-26 11:58:03,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-26 11:58:03,009 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1086125432:90 , srv1156869425:90 , srv1160056597:90 , srv1239798910:90 , srv1288161943:90 , srv12887284:90 , srv1306968728:90 , srv1334931806:90 , srv138351398:90 , srv1503596119:90 , srv1508491888:90 , srv1514141767:90 , srv1547772868:90 , srv1579684215:90 , srv1598113064:90 , srv1673540513:90 , srv1894043367:90 , srv2001695170:90 , srv2051433281:90 , srv383514237:90 , srv420391003:90 , srv45522112:90 , srv478257586:90 , srv521967606:90 , srv586763432:90 , srv674356432:90 , srv734704649:90 , srv813502877:90 , srv814974893:90 , srv967150906:90 }
2023-03-26 11:58:03,021 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 262), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 361), ProcessCount=169 (was 169), AvailableMemoryMB=9193 (was 9043) - AvailableMemoryMB LEAK? -