2023-07-16 00:53:14,454 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-16 00:53:14,758 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-16 00:53:14,870 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-16 00:53:14,937 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-16 00:53:14,946 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-16 00:53:14,958 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=262, ProcessCount=172, AvailableMemoryMB=7598
2023-07-16 00:53:14,960 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-16 00:53:14,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-16 00:53:15,120 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006026921:84 , srv1030732985:84 , srv103259656:84 , srv1039080928:84 , srv1062554939:84 , srv1080823083:84 , srv1089822534:84 , srv1112999991:84 , srv1121739339:84 , srv1125272178:84 , srv1154294990:84 , srv1205134609:84 , srv1208334889:84 , srv125492408:84 , srv12590904:84 , srv1265986000:84 , srv1271276271:84 , srv1310544677:84 , srv1310582595:84 , srv1338006242:84 , srv1350573314:84 , srv1354839502:84 , srv1356006729:84 , srv1408797934:84 , srv1429955844:84 , srv1460765797:84 , srv1486667238:84 , srv1489465601:84 , srv153542404:84 , srv1563288622:84 , srv1590247532:84 , srv1601256915:84 , srv1609554972:84 , srv1645036321:84 , srv1651232731:84 , srv1682550439:84 , srv1754498931:84 , srv1758720922:84 , srv1776322353:84 , srv1795217400:84 , srv1829188002:84 , srv1861093745:84 , srv1935067647:84 , srv195781004:84 , srv1982088752:84 , srv1999987801:84 , srv2001124547:84 , srv2003608763:84 , srv2049476752:84 , srv2051436041:84 , srv2062499154:84 , srv2094593529:84 , srv211785968:84 , srv2135374680:84 , srv2137137499:84 , srv221792745:84 , srv237378293:84 , srv250300847:84 , srv262533891:84 , srv269531490:84 , srv316173266:84 , srv321290363:84 , srv337852336:84 , srv367942419:84 , srv394569927:84 , srv420405376:84 , srv445094317:84 , srv465205338:84 , srv507274152:84 , srv510390922:84 , srv525194595:84 , srv536322863:84 , srv550677791:84 , srv553343569:84 , srv561031348:84 , srv596144624:84 , srv637277544:84 , srv656779208:84 , srv719381346:84 , srv728684578:84 , srv729459258:84 , srv738771409:84 , srv74232542:84 , srv746244237:84 , srv749001699:84 , srv751319217:84 , srv781137870:84 , srv785944399:84 , srv786949366:84 , srv81955182:84 , srv835817148:84 , srv858940169:84 , srv880503368:84 , srv882166936:84 , srv91070632:84 , srv929624142:84 , srv951669241:84 , srv976434852:84 , srv992610690:84 , srv232113146:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-16 00:53:15,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:15,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746244237=84, srv1408797934=23, srv1590247532=30, srv835817148=91, srv1601256915=31, srv858940169=92, srv550677791=73, srv1112999991=7, srv1310582595=18, srv2062499154=50, srv211785968=52, srv2137137499=54, srv1208334889=12, srv728684578=80, srv103259656=2, srv262533891=59, srv237378293=57, srv2051436041=49, srv1563288622=29, srv1039080928=3, srv2049476752=48, srv2135374680=53, srv2094593529=51, srv1429955844=24, srv749001699=85, srv1982088752=44, srv269531490=60, srv510390922=70, srv1089822534=6, srv1645036321=33, srv1999987801=45, srv880503368=93, srv561031348=75, srv91070632=95, srv507274152=69, srv420405376=66, srv2001124547=46, srv536322863=72, srv929624142=96, srv1861093745=41, srv321290363=62, srv1030732985=1, srv1754498931=36, srv153542404=28, srv337852336=63, srv2003608763=47, srv1121739339=8, srv1682550439=35, srv1062554939=4, srv951669241=97, srv81955182=90, srv1829188002=40, srv738771409=82, srv1486667238=26, srv1080823083=5, srv221792745=55, srv12590904=14, srv316173266=61, srv1310544677=17, srv1776322353=38, srv525194595=71, srv74232542=83, srv785944399=88, srv596144624=76, srv637277544=77, srv719381346=79, srv195781004=43, srv729459258=81, srv992610690=99, srv232113146=56, srv1154294990=10, srv1609554972=32, srv751319217=86, srv976434852=98, srv1350573314=20, srv250300847=58, srv1354839502=21, srv445094317=67, srv1271276271=16, srv465205338=68, srv1795217400=39, srv882166936=94, srv1935067647=42, srv1338006242=19, srv786949366=89, srv1651232731=34, srv1489465601=27, srv125492408=13, srv656779208=78, srv1265986000=15, srv1460765797=25, srv1125272178=9, srv553343569=74, srv1356006729=22, srv367942419=64, srv1006026921=0, srv394569927=65, srv1758720922=37, srv781137870=87, srv1205134609=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:15,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:15,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:15,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:15,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:15,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:15,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:15,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:15,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:15,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:15,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:15,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:15,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:15,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:15,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:15,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-16 00:53:15,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-16 00:53:15,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-16 00:53:15,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-16 00:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-16 00:53:15,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-16 00:53:15,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-16 00:53:15,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-16 00:53:15,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-16 00:53:15,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-16 00:53:15,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-16 00:53:15,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:15,465 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-16 00:53:15,465 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-16 00:53:16,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1170 ms to try 5000 different iterations.  Found a solution that moves 4661 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1836148156719939. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039718425332415236, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0510166012460023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5178888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039718425332415236, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.247, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:16,471 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv74232542:81 , srv1590247532:82 , srv1601256915:82 , srv2049476752:82 , srv1154294990:83 , srv1205134609:83 , srv1682550439:83 , srv1935067647:83 , srv250300847:83 , srv510390922:83 , srv91070632:83 , srv951669241:83 , srv1112999991:84 , srv1121739339:84 , srv1265986000:84 , srv1829188002:84 , srv2094593529:84 , srv316173266:84 , srv525194595:84 , srv561031348:84 , srv637277544:84 , srv786949366:84 , srv81955182:84 , srv1039080928:85 , srv1354839502:85 , srv1651232731:85 , srv1754498931:85 , srv1776322353:85 , srv1795217400:85 , srv1999987801:85 , srv2003608763:85 , srv221792745:85 , srv237378293:85 , srv507274152:85 , srv553343569:85 , srv976434852:85 , srv1089822534:86 , srv12590904:86 , srv1271276271:86 , srv1356006729:86 , srv1408797934:86 , srv1489465601:86 , srv153542404:86 , srv1861093745:86 , srv195781004:86 , srv2001124547:86 , srv2062499154:86 , srv321290363:86 , srv337852336:86 , srv536322863:86 , srv596144624:86 , srv738771409:86 , srv746244237:86 , srv835817148:86 , srv929624142:86 , srv1006026921:87 , srv1125272178:87 , srv1310544677:87 , srv1310582595:87 , srv1429955844:87 , srv1609554972:87 , srv2051436041:87 , srv269531490:87 , srv394569927:87 , srv420405376:87 , srv550677791:87 , srv719381346:87 , srv728684578:87 , srv858940169:87 , srv882166936:87 , srv1338006242:88 , srv1350573314:88 , srv1645036321:88 , srv2137137499:88 , srv367942419:88 , srv445094317:88 , srv880503368:88 , srv992610690:88 , srv103259656:89 , srv1080823083:89 , srv1460765797:89 , srv1486667238:89 , srv262533891:89 , srv465205338:89 , srv749001699:89 , srv751319217:89 , srv785944399:89 , srv1030732985:90 , srv1062554939:90 , srv1758720922:90 , srv781137870:90 , srv1208334889:91 , srv125492408:91 , srv1982088752:91 , srv656779208:91 , srv729459258:91 , srv211785968:92 , srv2135374680:92 , srv1563288622:94 , srv232113146:443 }
2023-07-16 00:53:16,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:16,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746244237=84, srv1408797934=23, srv1590247532=30, srv835817148=91, srv1601256915=31, srv858940169=92, srv550677791=73, srv1112999991=7, srv1310582595=18, srv2062499154=50, srv211785968=52, srv2137137499=54, srv1208334889=12, srv728684578=80, srv103259656=2, srv262533891=59, srv237378293=57, srv2051436041=49, srv1563288622=29, srv1039080928=3, srv2049476752=48, srv2135374680=53, srv2094593529=51, srv1429955844=24, srv749001699=85, srv1982088752=44, srv269531490=60, srv510390922=70, srv1089822534=6, srv1645036321=33, srv1999987801=45, srv880503368=93, srv561031348=75, srv91070632=95, srv507274152=69, srv420405376=66, srv2001124547=46, srv536322863=72, srv929624142=96, srv1861093745=41, srv321290363=62, srv1030732985=1, srv1754498931=36, srv153542404=28, srv337852336=63, srv2003608763=47, srv1121739339=8, srv1682550439=35, srv1062554939=4, srv951669241=97, srv81955182=90, srv1829188002=40, srv738771409=82, srv1486667238=26, srv1080823083=5, srv221792745=55, srv12590904=14, srv316173266=61, srv1310544677=17, srv1776322353=38, srv525194595=71, srv74232542=83, srv785944399=88, srv596144624=76, srv637277544=77, srv719381346=79, srv195781004=43, srv729459258=81, srv992610690=99, srv232113146=56, srv1154294990=10, srv1609554972=32, srv751319217=86, srv976434852=98, srv1350573314=20, srv250300847=58, srv1354839502=21, srv445094317=67, srv1271276271=16, srv465205338=68, srv1795217400=39, srv882166936=94, srv1935067647=42, srv1338006242=19, srv786949366=89, srv1651232731=34, srv1489465601=27, srv125492408=13, srv656779208=78, srv1265986000=15, srv1460765797=25, srv1125272178=9, srv553343569=74, srv1356006729=22, srv367942419=64, srv1006026921=0, srv394569927=65, srv1758720922=37, srv781137870=87, srv1205134609=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-16 00:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-16 00:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-16 00:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-16 00:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-16 00:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-16 00:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-16 00:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-16 00:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-16 00:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-16 00:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-16 00:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-16 00:53:16,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039718425332415236, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0510166012460023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039718425332415236, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.247, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:16,690 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-16 00:53:16,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1835821786524289, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039718425332415236, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0510166012460023, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039718425332415236, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17875, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.247, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-16 00:53:17,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 2655 regions; Going from a computed imbalance of 0.1835821786524289 to a new imbalance of 0.01387143268169279. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019378676654158874); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031729071948829624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.295, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019378676654158874); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:17,055 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1982088752:79 , srv1208334889:80 , srv211785968:80 , srv2135374680:81 , srv729459258:81 , srv103259656:82 , srv1062554939:82 , srv125492408:82 , srv1486667238:82 , srv1758720922:82 , srv445094317:82 , srv749001699:82 , srv785944399:82 , srv1080823083:83 , srv1350573314:83 , srv1429955844:83 , srv1563288622:83 , srv1609554972:83 , srv262533891:83 , srv367942419:83 , srv781137870:83 , srv858940169:83 , srv1006026921:84 , srv1030732985:84 , srv1125272178:84 , srv1338006242:84 , srv1460765797:84 , srv465205338:84 , srv746244237:84 , srv751319217:84 , srv929624142:84 , srv992610690:84 , srv12590904:85 , srv1310544677:85 , srv1408797934:85 , srv1489465601:85 , srv1645036321:85 , srv1754498931:85 , srv1776322353:85 , srv2001124547:85 , srv2137137499:85 , srv269531490:85 , srv321290363:85 , srv420405376:85 , srv882166936:85 , srv976434852:85 , srv1310582595:86 , srv1356006729:86 , srv153542404:86 , srv1795217400:86 , srv1829188002:86 , srv1861093745:86 , srv2094593529:86 , srv221792745:86 , srv337852336:86 , srv394569927:86 , srv550677791:86 , srv596144624:86 , srv656779208:86 , srv835817148:86 , srv880503368:86 , srv1039080928:87 , srv1089822534:87 , srv1112999991:87 , srv1354839502:87 , srv195781004:87 , srv2051436041:87 , srv250300847:87 , srv525194595:87 , srv536322863:87 , srv553343569:87 , srv719381346:87 , srv728684578:87 , srv738771409:87 , srv1121739339:88 , srv1205134609:88 , srv1265986000:88 , srv1271276271:88 , srv2003608763:88 , srv637277544:88 , srv1651232731:89 , srv2062499154:89 , srv237378293:89 , srv510390922:89 , srv81955182:89 , srv1154294990:90 , srv1590247532:90 , srv507274152:90 , srv561031348:90 , srv786949366:90 , srv951669241:90 , srv1601256915:91 , srv1682550439:91 , srv1935067647:91 , srv1999987801:91 , srv2049476752:91 , srv316173266:91 , srv91070632:91 , srv74232542:92 , srv232113146:503 }
2023-07-16 00:53:17,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:17,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746244237=84, srv1408797934=23, srv1590247532=30, srv835817148=91, srv1601256915=31, srv858940169=92, srv550677791=73, srv1112999991=7, srv1310582595=18, srv2062499154=50, srv211785968=52, srv2137137499=54, srv1208334889=12, srv728684578=80, srv103259656=2, srv262533891=59, srv237378293=57, srv2051436041=49, srv1563288622=29, srv1039080928=3, srv2049476752=48, srv2135374680=53, srv2094593529=51, srv1429955844=24, srv749001699=85, srv1982088752=44, srv269531490=60, srv510390922=70, srv1089822534=6, srv1645036321=33, srv1999987801=45, srv880503368=93, srv561031348=75, srv91070632=95, srv507274152=69, srv420405376=66, srv2001124547=46, srv536322863=72, srv929624142=96, srv1861093745=41, srv321290363=62, srv1030732985=1, srv1754498931=36, srv153542404=28, srv337852336=63, srv2003608763=47, srv1121739339=8, srv1682550439=35, srv1062554939=4, srv951669241=97, srv81955182=90, srv1829188002=40, srv738771409=82, srv1486667238=26, srv1080823083=5, srv221792745=55, srv12590904=14, srv316173266=61, srv1310544677=17, srv1776322353=38, srv525194595=71, srv74232542=83, srv785944399=88, srv596144624=76, srv637277544=77, srv719381346=79, srv195781004=43, srv729459258=81, srv992610690=99, srv232113146=56, srv1154294990=10, srv1609554972=32, srv751319217=86, srv976434852=98, srv1350573314=20, srv250300847=58, srv1354839502=21, srv445094317=67, srv1271276271=16, srv465205338=68, srv1795217400=39, srv882166936=94, srv1935067647=42, srv1338006242=19, srv786949366=89, srv1651232731=34, srv1489465601=27, srv125492408=13, srv656779208=78, srv1265986000=15, srv1460765797=25, srv1125272178=9, srv553343569=74, srv1356006729=22, srv367942419=64, srv1006026921=0, srv394569927=65, srv1758720922=37, srv781137870=87, srv1205134609=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:17,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:17,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:17,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:17,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:17,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:17,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:17,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:17,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-16 00:53:17,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-16 00:53:17,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-16 00:53:17,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-16 00:53:17,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-16 00:53:17,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-16 00:53:17,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-16 00:53:17,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-16 00:53:17,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-16 00:53:17,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-16 00:53:17,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-16 00:53:17,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-16 00:53:17,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-16 00:53:17,209 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019378676654158874); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031729071948829624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019378676654158874); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:17,210 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-16 00:53:17,210 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013852841974345635, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019378676654158874); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031729071948829624, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019378676654158874); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-16 00:53:17,524 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 419 ms to try 5000 different iterations.  Found a solution that moves 1026 regions; Going from a computed imbalance of 0.013852841974345635 to a new imbalance of 3.960208943451355E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0032288140322523505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004593461414505888); MoveCostFunction : (multiplier=7.0, imbalance=0.114, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0032288140322523505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:17,526 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv656779208:80 , srv1563288622:82 , srv1999987801:82 , srv507274152:82 , srv1271276271:83 , srv1935067647:83 , srv2051436041:83 , srv316173266:83 , srv561031348:83 , srv719381346:83 , srv728684578:83 , srv880503368:83 , srv1030732985:84 , srv1154294990:84 , srv125492408:84 , srv1310582595:84 , srv1601256915:84 , srv1651232731:84 , srv2049476752:84 , srv2062499154:84 , srv237378293:84 , srv781137870:84 , srv786949366:84 , srv91070632:84 , srv1089822534:85 , srv1121739339:85 , srv1338006242:85 , srv1354839502:85 , srv1460765797:85 , srv1645036321:85 , srv1682550439:85 , srv195781004:85 , srv2003608763:85 , srv2135374680:85 , srv367942419:85 , srv394569927:85 , srv420405376:85 , srv465205338:85 , srv510390922:85 , srv536322863:85 , srv596144624:85 , srv751319217:85 , srv951669241:85 , srv1006026921:86 , srv103259656:86 , srv1039080928:86 , srv1062554939:86 , srv1080823083:86 , srv1112999991:86 , srv1125272178:86 , srv1208334889:86 , srv1265986000:86 , srv1310544677:86 , srv1350573314:86 , srv1356006729:86 , srv1408797934:86 , srv1486667238:86 , srv1590247532:86 , srv1758720922:86 , srv1795217400:86 , srv1861093745:86 , srv211785968:86 , srv2137137499:86 , srv221792745:86 , srv262533891:86 , srv269531490:86 , srv337852336:86 , srv525194595:86 , srv550677791:86 , srv553343569:86 , srv637277544:86 , srv729459258:86 , srv738771409:86 , srv74232542:86 , srv785944399:86 , srv81955182:86 , srv835817148:86 , srv1205134609:87 , srv12590904:87 , srv1429955844:87 , srv1489465601:87 , srv153542404:87 , srv2001124547:87 , srv321290363:87 , srv749001699:87 , srv858940169:87 , srv882166936:87 , srv929624142:87 , srv976434852:87 , srv992610690:87 , srv1609554972:88 , srv1754498931:88 , srv1776322353:88 , srv1829188002:88 , srv1982088752:88 , srv2094593529:88 , srv250300847:88 , srv445094317:88 , srv746244237:88 , srv232113146:540 }
2023-07-16 00:53:17,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:17,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746244237=84, srv1408797934=23, srv1590247532=30, srv835817148=91, srv1601256915=31, srv858940169=92, srv550677791=73, srv1112999991=7, srv1310582595=18, srv2062499154=50, srv211785968=52, srv2137137499=54, srv1208334889=12, srv728684578=80, srv103259656=2, srv262533891=59, srv237378293=57, srv2051436041=49, srv1563288622=29, srv1039080928=3, srv2049476752=48, srv2135374680=53, srv2094593529=51, srv1429955844=24, srv749001699=85, srv1982088752=44, srv269531490=60, srv510390922=70, srv1089822534=6, srv1645036321=33, srv1999987801=45, srv880503368=93, srv561031348=75, srv91070632=95, srv507274152=69, srv420405376=66, srv2001124547=46, srv536322863=72, srv929624142=96, srv1861093745=41, srv321290363=62, srv1030732985=1, srv1754498931=36, srv153542404=28, srv337852336=63, srv2003608763=47, srv1121739339=8, srv1682550439=35, srv1062554939=4, srv951669241=97, srv81955182=90, srv1829188002=40, srv738771409=82, srv1486667238=26, srv1080823083=5, srv221792745=55, srv12590904=14, srv316173266=61, srv1310544677=17, srv1776322353=38, srv525194595=71, srv74232542=83, srv785944399=88, srv596144624=76, srv637277544=77, srv719381346=79, srv195781004=43, srv729459258=81, srv992610690=99, srv232113146=56, srv1154294990=10, srv1609554972=32, srv751319217=86, srv976434852=98, srv1350573314=20, srv250300847=58, srv1354839502=21, srv445094317=67, srv1271276271=16, srv465205338=68, srv1795217400=39, srv882166936=94, srv1935067647=42, srv1338006242=19, srv786949366=89, srv1651232731=34, srv1489465601=27, srv125492408=13, srv656779208=78, srv1265986000=15, srv1460765797=25, srv1125272178=9, srv553343569=74, srv1356006729=22, srv367942419=64, srv1006026921=0, srv394569927=65, srv1758720922=37, srv781137870=87, srv1205134609=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:17,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:17,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-16 00:53:17,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-16 00:53:17,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-16 00:53:17,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-16 00:53:17,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-16 00:53:17,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-16 00:53:17,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-16 00:53:17,685 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0032288140322523505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004593461414505888); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0032288140322523505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:17,685 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-16 00:53:17,686 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.8883668879403135E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0032288140322523505); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004593461414505888); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0032288140322523505); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-16 00:53:17,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 401 ms to try 5000 different iterations.  Found a solution that moves 192 regions; Going from a computed imbalance of 3.8883668879403135E-4 to a new imbalance of 1.9161204996519276E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.021333333333333333); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:17,971 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv153542404:83 , srv1563288622:83 , srv1682550439:83 , srv2062499154:83 , srv2137137499:83 , srv465205338:83 , srv550677791:83 , srv738771409:83 , srv74232542:83 , srv81955182:83 , srv882166936:83 , srv992610690:83 , srv1030732985:84 , srv1039080928:84 , srv1062554939:84 , srv1080823083:84 , srv1089822534:84 , srv125492408:84 , srv12590904:84 , srv1265986000:84 , srv1310544677:84 , srv1356006729:84 , srv1460765797:84 , srv1489465601:84 , srv1590247532:84 , srv1609554972:84 , srv1645036321:84 , srv1651232731:84 , srv1754498931:84 , srv1758720922:84 , srv1776322353:84 , srv1829188002:84 , srv1861093745:84 , srv195781004:84 , srv1982088752:84 , srv1999987801:84 , srv2001124547:84 , srv2094593529:84 , srv211785968:84 , srv2135374680:84 , srv237378293:84 , srv250300847:84 , srv262533891:84 , srv269531490:84 , srv316173266:84 , srv321290363:84 , srv337852336:84 , srv394569927:84 , srv445094317:84 , srv536322863:84 , srv553343569:84 , srv596144624:84 , srv637277544:84 , srv729459258:84 , srv746244237:84 , srv749001699:84 , srv751319217:84 , srv835817148:84 , srv858940169:84 , srv951669241:84 , srv1006026921:85 , srv103259656:85 , srv1112999991:85 , srv1121739339:85 , srv1125272178:85 , srv1154294990:85 , srv1205134609:85 , srv1208334889:85 , srv1271276271:85 , srv1310582595:85 , srv1338006242:85 , srv1350573314:85 , srv1354839502:85 , srv1408797934:85 , srv1429955844:85 , srv1486667238:85 , srv1601256915:85 , srv1795217400:85 , srv1935067647:85 , srv2003608763:85 , srv2049476752:85 , srv2051436041:85 , srv221792745:85 , srv420405376:85 , srv507274152:85 , srv510390922:85 , srv525194595:85 , srv561031348:85 , srv656779208:85 , srv719381346:85 , srv728684578:85 , srv781137870:85 , srv785944399:85 , srv786949366:85 , srv880503368:85 , srv91070632:85 , srv929624142:85 , srv976434852:85 , srv367942419:86 , srv232113146:656 }
2023-07-16 00:53:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv746244237=84, srv1408797934=23, srv1590247532=30, srv835817148=91, srv1601256915=31, srv858940169=92, srv550677791=73, srv1112999991=7, srv1310582595=18, srv2062499154=50, srv211785968=52, srv2137137499=54, srv1208334889=12, srv728684578=80, srv103259656=2, srv262533891=59, srv237378293=57, srv2051436041=49, srv1563288622=29, srv1039080928=3, srv2049476752=48, srv2135374680=53, srv2094593529=51, srv1429955844=24, srv749001699=85, srv1982088752=44, srv269531490=60, srv510390922=70, srv1089822534=6, srv1645036321=33, srv1999987801=45, srv880503368=93, srv561031348=75, srv91070632=95, srv507274152=69, srv420405376=66, srv2001124547=46, srv536322863=72, srv929624142=96, srv1861093745=41, srv321290363=62, srv1030732985=1, srv1754498931=36, srv153542404=28, srv337852336=63, srv2003608763=47, srv1121739339=8, srv1682550439=35, srv1062554939=4, srv951669241=97, srv81955182=90, srv1829188002=40, srv738771409=82, srv1486667238=26, srv1080823083=5, srv221792745=55, srv12590904=14, srv316173266=61, srv1310544677=17, srv1776322353=38, srv525194595=71, srv74232542=83, srv785944399=88, srv596144624=76, srv637277544=77, srv719381346=79, srv195781004=43, srv729459258=81, srv992610690=99, srv232113146=56, srv1154294990=10, srv1609554972=32, srv751319217=86, srv976434852=98, srv1350573314=20, srv250300847=58, srv1354839502=21, srv445094317=67, srv1271276271=16, srv465205338=68, srv1795217400=39, srv882166936=94, srv1935067647=42, srv1338006242=19, srv786949366=89, srv1651232731=34, srv1489465601=27, srv125492408=13, srv656779208=78, srv1265986000=15, srv1460765797=25, srv1125272178=9, srv553343569=74, srv1356006729=22, srv367942419=64, srv1006026921=0, srv394569927=65, srv1758720922=37, srv781137870=87, srv1205134609=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:17,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:17,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:17,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:17,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:17,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-16 00:53:17,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-16 00:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-16 00:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-16 00:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-16 00:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-16 00:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-16 00:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-16 00:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-16 00:53:18,107 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.781679226180972E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:18,107 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv153542404:83 , srv1563288622:83 , srv1682550439:83 , srv2062499154:83 , srv2137137499:83 , srv465205338:83 , srv550677791:83 , srv738771409:83 , srv74232542:83 , srv81955182:83 , srv882166936:83 , srv992610690:83 , srv1030732985:84 , srv1039080928:84 , srv1062554939:84 , srv1080823083:84 , srv1089822534:84 , srv125492408:84 , srv12590904:84 , srv1265986000:84 , srv1310544677:84 , srv1356006729:84 , srv1460765797:84 , srv1489465601:84 , srv1590247532:84 , srv1609554972:84 , srv1645036321:84 , srv1651232731:84 , srv1754498931:84 , srv1758720922:84 , srv1776322353:84 , srv1829188002:84 , srv1861093745:84 , srv195781004:84 , srv1982088752:84 , srv1999987801:84 , srv2001124547:84 , srv2094593529:84 , srv211785968:84 , srv2135374680:84 , srv237378293:84 , srv250300847:84 , srv262533891:84 , srv269531490:84 , srv316173266:84 , srv321290363:84 , srv337852336:84 , srv394569927:84 , srv445094317:84 , srv536322863:84 , srv553343569:84 , srv596144624:84 , srv637277544:84 , srv729459258:84 , srv746244237:84 , srv749001699:84 , srv751319217:84 , srv835817148:84 , srv858940169:84 , srv951669241:84 , srv1006026921:85 , srv103259656:85 , srv1112999991:85 , srv1121739339:85 , srv1125272178:85 , srv1154294990:85 , srv1205134609:85 , srv1208334889:85 , srv1271276271:85 , srv1310582595:85 , srv1338006242:85 , srv1350573314:85 , srv1354839502:85 , srv1408797934:85 , srv1429955844:85 , srv1486667238:85 , srv1601256915:85 , srv1795217400:85 , srv1935067647:85 , srv2003608763:85 , srv2049476752:85 , srv2051436041:85 , srv221792745:85 , srv420405376:85 , srv507274152:85 , srv510390922:85 , srv525194595:85 , srv561031348:85 , srv656779208:85 , srv719381346:85 , srv728684578:85 , srv781137870:85 , srv785944399:85 , srv786949366:85 , srv880503368:85 , srv91070632:85 , srv929624142:85 , srv976434852:85 , srv367942419:86 , srv232113146:656 }
2023-07-16 00:53:18,131 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=313 (was 262) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=7003 (was 7598)
2023-07-16 00:53:18,143 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=172, AvailableMemoryMB=7002
2023-07-16 00:53:18,144 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-16 00:53:18,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-16 00:53:18,156 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1182326120:84 , srv1221816652:84 , srv1279572218:84 , srv1331903674:84 , srv1349198178:84 , srv1351601628:84 , srv1387223688:84 , srv1462320266:84 , srv1617267787:84 , srv1632648757:84 , srv1681904897:84 , srv1844605845:84 , srv1859125881:84 , srv1896030483:84 , srv1939153783:84 , srv1939310466:84 , srv2035704889:84 , srv228510648:84 , srv2676090:84 , srv363449598:84 , srv548328090:84 , srv5842913:84 , srv61393616:84 , srv68708916:84 , srv779863787:84 , srv791495349:84 , srv822642608:84 , srv901259754:84 , srv978720236:84 , srv1476955861:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-16 00:53:18,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:53:18,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1279572218=2, srv68708916=24, srv1632648757=10, srv1182326120=0, srv1349198178=4, srv1387223688=6, srv1331903674=3, srv2676090=19, srv779863787=25, srv1896030483=14, srv1221816652=1, srv2035704889=17, srv1462320266=7, srv1476955861=8, srv1939153783=15, srv363449598=20, srv822642608=27, srv978720236=29, srv228510648=18, srv5842913=22, srv61393616=23, srv1617267787=9, srv1844605845=12, srv548328090=21, srv1351601628=5, srv1681904897=11, srv1939310466=16, srv1859125881=13, srv901259754=28, srv791495349=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:53:18,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:53:18,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:53:18,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:53:18,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:53:18,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:53:18,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-16 00:53:18,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.180976347684808, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:53:18,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912763293826412, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.180976347684808, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-16 00:55:18,172 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6091961 different iterations.  Found a solution that moves 1857 regions; Going from a computed imbalance of 0.9912763293826412 to a new imbalance of 4.334330639506329E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:55:18,178 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1182326120:90 , srv1221816652:90 , srv1279572218:90 , srv1331903674:90 , srv1349198178:90 , srv1351601628:90 , srv1387223688:90 , srv1462320266:90 , srv1476955861:90 , srv1617267787:90 , srv1632648757:90 , srv1681904897:90 , srv1844605845:90 , srv1859125881:90 , srv1896030483:90 , srv1939153783:90 , srv1939310466:90 , srv2035704889:90 , srv228510648:90 , srv2676090:90 , srv363449598:90 , srv548328090:90 , srv5842913:90 , srv61393616:90 , srv68708916:90 , srv779863787:90 , srv791495349:90 , srv822642608:90 , srv901259754:90 , srv978720236:90 }
2023-07-16 00:55:18,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-16 00:55:18,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1279572218=2, srv68708916=24, srv1632648757=10, srv1182326120=0, srv1349198178=4, srv1387223688=6, srv1331903674=3, srv2676090=19, srv779863787=25, srv1896030483=14, srv1221816652=1, srv2035704889=17, srv1462320266=7, srv1476955861=8, srv1939153783=15, srv363449598=20, srv822642608=27, srv978720236=29, srv228510648=18, srv5842913=22, srv61393616=23, srv1617267787=9, srv1844605845=12, srv548328090=21, srv1351601628=5, srv1681904897=11, srv1939310466=16, srv1859125881=13, srv901259754=28, srv791495349=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-16 00:55:18,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-16 00:55:18,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-16 00:55:18,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-16 00:55:18,195 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-16 00:55:18,195 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1182326120:90 , srv1221816652:90 , srv1279572218:90 , srv1331903674:90 , srv1349198178:90 , srv1351601628:90 , srv1387223688:90 , srv1462320266:90 , srv1476955861:90 , srv1617267787:90 , srv1632648757:90 , srv1681904897:90 , srv1844605845:90 , srv1859125881:90 , srv1896030483:90 , srv1939153783:90 , srv1939310466:90 , srv2035704889:90 , srv228510648:90 , srv2676090:90 , srv363449598:90 , srv548328090:90 , srv5842913:90 , srv61393616:90 , srv68708916:90 , srv779863787:90 , srv791495349:90 , srv822642608:90 , srv901259754:90 , srv978720236:90 }
2023-07-16 00:55:18,207 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=238 (was 313), ProcessCount=170 (was 172), AvailableMemoryMB=6241 (was 7002)