2023-11-25 15:53:15,698 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-25 15:53:16,028 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-25 15:53:16,145 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-25 15:53:16,217 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 15:53:16,224 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-25 15:53:16,238 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=172, AvailableMemoryMB=9696
2023-11-25 15:53:16,241 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 15:53:16,241 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-25 15:53:16,378 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004085584:84 , srv1007469890:84 , srv1053228072:84 , srv1100391391:84 , srv1126450595:84 , srv1165233451:84 , srv1170766598:84 , srv1180008464:84 , srv1187148906:84 , srv1192488014:84 , srv1196742950:84 , srv1200857306:84 , srv1219100423:84 , srv1230849139:84 , srv1239206781:84 , srv1299465488:84 , srv1299799088:84 , srv1302923512:84 , srv1334692077:84 , srv1351212083:84 , srv1409369583:84 , srv146754236:84 , srv1483279300:84 , srv1535563919:84 , srv1644545990:84 , srv1702203684:84 , srv1706631351:84 , srv1712939228:84 , srv172878102:84 , srv1729870567:84 , srv1733869849:84 , srv1776611691:84 , srv1789102388:84 , srv1809578219:84 , srv1815672011:84 , srv1816667212:84 , srv1840174353:84 , srv1851483568:84 , srv1870268183:84 , srv1885521601:84 , srv1900080460:84 , srv1937843031:84 , srv1939313195:84 , srv1989217915:84 , srv1993985333:84 , srv1999489407:84 , srv2010176502:84 , srv2034506011:84 , srv2046833492:84 , srv2081987715:84 , srv2094413480:84 , srv2110472212:84 , srv2111695389:84 , srv2140044215:84 , srv221741840:84 , srv247507718:84 , srv250666242:84 , srv264040763:84 , srv276824228:84 , srv304836606:84 , srv343892433:84 , srv352098288:84 , srv356437714:84 , srv358819637:84 , srv454985016:84 , srv467156153:84 , srv476677145:84 , srv503151877:84 , srv507068607:84 , srv513664882:84 , srv522198109:84 , srv566687582:84 , srv603363600:84 , srv607510662:84 , srv615731444:84 , srv629207411:84 , srv635656730:84 , srv636979114:84 , srv698709161:84 , srv713983562:84 , srv739583364:84 , srv74811395:84 , srv74965688:84 , srv779735741:84 , srv795185943:84 , srv811814816:84 , srv81211775:84 , srv81741017:84 , srv827156362:84 , srv854913722:84 , srv857381877:84 , srv869104900:84 , srv907698068:84 , srv920199117:84 , srv936549343:84 , srv971058899:84 , srv973784634:84 , srv977658306:84 , srv994446811:84 , srv1465634968:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-25 15:53:16,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:16,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1900080460=41, srv1885521601=40, srv1939313195=43, srv2081987715=50, srv1239206781=14, srv356437714=63, srv81741017=88, srv503151877=68, srv607510662=74, srv81211775=87, srv869104900=92, srv629207411=76, srv1733869849=31, srv352098288=62, srv454985016=65, srv698709161=79, srv1851483568=38, srv507068607=69, srv467156153=66, srv513664882=70, srv1302923512=17, srv907698068=93, srv1776611691=32, srv74811395=82, srv1230849139=13, srv1870268183=39, srv1993985333=45, srv971058899=96, srv1789102388=33, srv1196742950=10, srv522198109=71, srv977658306=98, srv358819637=64, srv476677145=67, srv2110472212=52, srv172878102=29, srv1840174353=37, srv1187148906=8, srv1100391391=3, srv636979114=78, srv713983562=80, srv74965688=83, srv779735741=84, srv1729870567=30, srv1053228072=2, srv1409369583=20, srv615731444=75, srv1644545990=25, srv1200857306=11, srv603363600=73, srv343892433=61, srv2046833492=49, srv635656730=77, srv2140044215=54, srv827156362=89, srv994446811=99, srv1465634968=21, srv1712939228=28, srv1180008464=7, srv146754236=22, srv2111695389=53, srv1999489407=46, srv2094413480=51, srv1809578219=34, srv264040763=58, srv1483279300=23, srv811814816=86, srv1004085584=0, srv1706631351=27, srv739583364=81, srv250666242=57, srv1126450595=4, srv1351212083=19, srv1989217915=44, srv1165233451=5, srv1937843031=42, srv247507718=56, srv1192488014=9, srv973784634=97, srv1334692077=18, srv795185943=85, srv276824228=59, srv1535563919=24, srv857381877=91, srv1007469890=1, srv1815672011=35, srv854913722=90, srv2034506011=48, srv1299465488=15, srv1702203684=26, srv1219100423=12, srv2010176502=47, srv221741840=55, srv936549343=95, srv1170766598=6, srv1816667212=36, srv304836606=60, srv566687582=72, srv920199117=94, srv1299799088=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:16,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:16,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:16,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:16,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:16,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:16,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 15:53:16,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 15:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 15:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 15:53:16,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 15:53:16,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 15:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 15:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 15:53:16,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 15:53:16,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 15:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 15:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 15:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 15:53:16,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 15:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 15:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 15:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 15:53:16,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 15:53:16,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 15:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 15:53:16,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 15:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 15:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 15:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 15:53:16,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 15:53:16,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 15:53:16,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 15:53:16,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 15:53:16,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 15:53:16,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 15:53:16,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 15:53:16,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 15:53:16,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 15:53:16,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 15:53:16,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:16,744 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 15:53:16,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 15:53:17,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 606 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17936551231837058. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849163786213687, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0582921932850343, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849163786213687, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:17,187 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1816667212:81 , srv636979114:81 , srv2010176502:82 , srv356437714:82 , srv503151877:82 , srv869104900:82 , srv1004085584:83 , srv1192488014:83 , srv352098288:83 , srv467156153:83 , srv854913722:83 , srv1351212083:84 , srv1702203684:84 , srv1789102388:84 , srv1885521601:84 , srv607510662:84 , srv81211775:84 , srv920199117:84 , srv971058899:84 , srv1053228072:85 , srv1299465488:85 , srv1334692077:85 , srv1535563919:85 , srv1815672011:85 , srv2140044215:85 , srv221741840:85 , srv250666242:85 , srv343892433:85 , srv513664882:85 , srv615731444:85 , srv811814816:85 , srv81741017:85 , srv1126450595:86 , srv1165233451:86 , srv1180008464:86 , srv1219100423:86 , srv1302923512:86 , srv1409369583:86 , srv1706631351:86 , srv1712939228:86 , srv1733869849:86 , srv1870268183:86 , srv1900080460:86 , srv1989217915:86 , srv2046833492:86 , srv2081987715:86 , srv2111695389:86 , srv247507718:86 , srv264040763:86 , srv276824228:86 , srv476677145:86 , srv522198109:86 , srv566687582:86 , srv74965688:86 , srv857381877:86 , srv936549343:86 , srv994446811:86 , srv1100391391:87 , srv1230849139:87 , srv1239206781:87 , srv1299799088:87 , srv1644545990:87 , srv1776611691:87 , srv1851483568:87 , srv1939313195:87 , srv358819637:87 , srv698709161:87 , srv795185943:87 , srv973784634:87 , srv977658306:87 , srv1170766598:88 , srv1993985333:88 , srv304836606:88 , srv629207411:88 , srv1187148906:89 , srv1200857306:89 , srv146754236:89 , srv1483279300:89 , srv172878102:89 , srv1729870567:89 , srv1840174353:89 , srv507068607:89 , srv713983562:89 , srv739583364:89 , srv74811395:89 , srv1007469890:90 , srv2034506011:90 , srv2110472212:90 , srv454985016:90 , srv779735741:90 , srv907698068:90 , srv1809578219:91 , srv1999489407:91 , srv1196742950:92 , srv1937843031:92 , srv2094413480:92 , srv603363600:92 , srv635656730:92 , srv827156362:92 , srv1465634968:432 }
2023-11-25 15:53:17,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:17,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1900080460=41, srv1885521601=40, srv1939313195=43, srv2081987715=50, srv1239206781=14, srv356437714=63, srv81741017=88, srv503151877=68, srv607510662=74, srv81211775=87, srv869104900=92, srv629207411=76, srv1733869849=31, srv352098288=62, srv454985016=65, srv698709161=79, srv1851483568=38, srv507068607=69, srv467156153=66, srv513664882=70, srv1302923512=17, srv907698068=93, srv1776611691=32, srv74811395=82, srv1230849139=13, srv1870268183=39, srv1993985333=45, srv971058899=96, srv1789102388=33, srv1196742950=10, srv522198109=71, srv977658306=98, srv358819637=64, srv476677145=67, srv2110472212=52, srv172878102=29, srv1840174353=37, srv1187148906=8, srv1100391391=3, srv636979114=78, srv713983562=80, srv74965688=83, srv779735741=84, srv1729870567=30, srv1053228072=2, srv1409369583=20, srv615731444=75, srv1644545990=25, srv1200857306=11, srv603363600=73, srv343892433=61, srv2046833492=49, srv635656730=77, srv2140044215=54, srv827156362=89, srv994446811=99, srv1465634968=21, srv1712939228=28, srv1180008464=7, srv146754236=22, srv2111695389=53, srv1999489407=46, srv2094413480=51, srv1809578219=34, srv264040763=58, srv1483279300=23, srv811814816=86, srv1004085584=0, srv1706631351=27, srv739583364=81, srv250666242=57, srv1126450595=4, srv1351212083=19, srv1989217915=44, srv1165233451=5, srv1937843031=42, srv247507718=56, srv1192488014=9, srv973784634=97, srv1334692077=18, srv795185943=85, srv276824228=59, srv1535563919=24, srv857381877=91, srv1007469890=1, srv1815672011=35, srv854913722=90, srv2034506011=48, srv1299465488=15, srv1702203684=26, srv1219100423=12, srv2010176502=47, srv221741840=55, srv936549343=95, srv1170766598=6, srv1816667212=36, srv304836606=60, srv566687582=72, srv920199117=94, srv1299799088=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:17,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:17,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:17,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:17,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:17,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 15:53:17,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 15:53:17,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 15:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 15:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 15:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 15:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 15:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 15:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 15:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 15:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 15:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 15:53:17,424 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849163786213687, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0582921932850343, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849163786213687, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:17,425 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 15:53:17,425 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17933270724721376, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849163786213687, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0582921932850343, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849163786213687, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 15:53:17,724 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 2652 regions; Going from a computed imbalance of 0.17933270724721376 to a new imbalance of 0.01577995249614429. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02184243362635071); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039749496181575854, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2946666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02184243362635071); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043583333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:17,739 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv635656730:79 , srv827156362:79 , srv1196742950:80 , srv1809578219:80 , srv1937843031:80 , srv1999489407:81 , srv2094413480:81 , srv454985016:81 , srv603363600:81 , srv713983562:81 , srv779735741:81 , srv1170766598:82 , srv146754236:82 , srv1483279300:82 , srv2110472212:82 , srv1007469890:83 , srv1840174353:83 , srv1993985333:83 , srv2034506011:83 , srv507068607:83 , srv907698068:83 , srv1187148906:84 , srv1230849139:84 , srv1299799088:84 , srv1302923512:84 , srv1706631351:84 , srv1989217915:84 , srv2046833492:84 , srv264040763:84 , srv304836606:84 , srv522198109:84 , srv739583364:84 , srv74811395:84 , srv1053228072:85 , srv1165233451:85 , srv1200857306:85 , srv1219100423:85 , srv1239206781:85 , srv1334692077:85 , srv1644545990:85 , srv1702203684:85 , srv1729870567:85 , srv1776611691:85 , srv1870268183:85 , srv1939313195:85 , srv476677145:85 , srv566687582:85 , srv698709161:85 , srv857381877:85 , srv936549343:85 , srv973784634:85 , srv1126450595:86 , srv1299465488:86 , srv1351212083:86 , srv1409369583:86 , srv1851483568:86 , srv1900080460:86 , srv607510662:86 , srv629207411:86 , srv811814816:86 , srv977658306:86 , srv1789102388:87 , srv1815672011:87 , srv2081987715:87 , srv2111695389:87 , srv247507718:87 , srv250666242:87 , srv276824228:87 , srv358819637:87 , srv467156153:87 , srv615731444:87 , srv74965688:87 , srv81741017:87 , srv854913722:87 , srv994446811:87 , srv1712939228:88 , srv1733869849:88 , srv1885521601:88 , srv2140044215:88 , srv221741840:88 , srv343892433:88 , srv356437714:88 , srv513664882:88 , srv795185943:88 , srv81211775:88 , srv920199117:88 , srv971058899:88 , srv1100391391:89 , srv1180008464:89 , srv1535563919:89 , srv352098288:89 , srv1004085584:90 , srv1192488014:90 , srv503151877:90 , srv172878102:91 , srv1816667212:91 , srv2010176502:91 , srv636979114:91 , srv869104900:92 , srv1465634968:536 }
2023-11-25 15:53:17,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:17,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1900080460=41, srv1885521601=40, srv1939313195=43, srv2081987715=50, srv1239206781=14, srv356437714=63, srv81741017=88, srv503151877=68, srv607510662=74, srv81211775=87, srv869104900=92, srv629207411=76, srv1733869849=31, srv352098288=62, srv454985016=65, srv698709161=79, srv1851483568=38, srv507068607=69, srv467156153=66, srv513664882=70, srv1302923512=17, srv907698068=93, srv1776611691=32, srv74811395=82, srv1230849139=13, srv1870268183=39, srv1993985333=45, srv971058899=96, srv1789102388=33, srv1196742950=10, srv522198109=71, srv977658306=98, srv358819637=64, srv476677145=67, srv2110472212=52, srv172878102=29, srv1840174353=37, srv1187148906=8, srv1100391391=3, srv636979114=78, srv713983562=80, srv74965688=83, srv779735741=84, srv1729870567=30, srv1053228072=2, srv1409369583=20, srv615731444=75, srv1644545990=25, srv1200857306=11, srv603363600=73, srv343892433=61, srv2046833492=49, srv635656730=77, srv2140044215=54, srv827156362=89, srv994446811=99, srv1465634968=21, srv1712939228=28, srv1180008464=7, srv146754236=22, srv2111695389=53, srv1999489407=46, srv2094413480=51, srv1809578219=34, srv264040763=58, srv1483279300=23, srv811814816=86, srv1004085584=0, srv1706631351=27, srv739583364=81, srv250666242=57, srv1126450595=4, srv1351212083=19, srv1989217915=44, srv1165233451=5, srv1937843031=42, srv247507718=56, srv1192488014=9, srv973784634=97, srv1334692077=18, srv795185943=85, srv276824228=59, srv1535563919=24, srv857381877=91, srv1007469890=1, srv1815672011=35, srv854913722=90, srv2034506011=48, srv1299465488=15, srv1702203684=26, srv1219100423=12, srv2010176502=47, srv221741840=55, srv936549343=95, srv1170766598=6, srv1816667212=36, srv304836606=60, srv566687582=72, srv920199117=94, srv1299799088=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:17,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:17,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:17,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:17,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 15:53:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 15:53:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 15:53:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 15:53:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 15:53:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 15:53:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 15:53:17,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 15:53:17,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 15:53:17,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 15:53:17,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 15:53:17,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 15:53:17,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 15:53:17,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02184243362635071); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039749496181575854, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02184243362635071); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043583333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:17,915 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 15:53:17,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015761382795246113, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02184243362635071); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039749496181575854, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02184243362635071); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043583333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 15:53:18,189 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 386 ms to try 5000 different iterations.  Found a solution that moves 948 regions; Going from a computed imbalance of 0.015761382795246113 to a new imbalance of 7.164128990349792E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00709966812198461); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01174934628169321); MoveCostFunction : (multiplier=7.0, imbalance=0.10533333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00709966812198461); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:18,191 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv172878102:77 , srv1100391391:82 , srv1180008464:82 , srv1007469890:83 , srv1200857306:83 , srv1535563919:83 , srv1729870567:83 , srv2140044215:83 , srv795185943:83 , srv869104900:83 , srv1187148906:84 , srv1192488014:84 , srv1712939228:84 , srv1733869849:84 , srv2010176502:84 , srv2081987715:84 , srv2094413480:84 , srv2111695389:84 , srv343892433:84 , srv358819637:84 , srv629207411:84 , srv74965688:84 , srv977658306:84 , srv994446811:84 , srv1004085584:85 , srv1196742950:85 , srv1409369583:85 , srv146754236:85 , srv1644545990:85 , srv1815672011:85 , srv1840174353:85 , srv1851483568:85 , srv1885521601:85 , srv1900080460:85 , srv1937843031:85 , srv1939313195:85 , srv2034506011:85 , srv221741840:85 , srv247507718:85 , srv250666242:85 , srv276824228:85 , srv352098288:85 , srv503151877:85 , srv507068607:85 , srv513664882:85 , srv603363600:85 , srv615731444:85 , srv636979114:85 , srv698709161:85 , srv739583364:85 , srv74811395:85 , srv827156362:85 , srv907698068:85 , srv1126450595:86 , srv1165233451:86 , srv1239206781:86 , srv1483279300:86 , srv1776611691:86 , srv1789102388:86 , srv1809578219:86 , srv1816667212:86 , srv1870268183:86 , srv1993985333:86 , srv1999489407:86 , srv2110472212:86 , srv454985016:86 , srv566687582:86 , srv635656730:86 , srv779735741:86 , srv81211775:86 , srv81741017:86 , srv857381877:86 , srv920199117:86 , srv936549343:86 , srv971058899:86 , srv973784634:86 , srv1053228072:87 , srv1170766598:87 , srv1219100423:87 , srv1230849139:87 , srv1299465488:87 , srv1299799088:87 , srv1302923512:87 , srv1334692077:87 , srv1351212083:87 , srv1706631351:87 , srv264040763:87 , srv304836606:87 , srv356437714:87 , srv467156153:87 , srv476677145:87 , srv522198109:87 , srv607510662:87 , srv811814816:87 , srv854913722:87 , srv1989217915:88 , srv2046833492:88 , srv713983562:88 , srv1702203684:89 , srv1465634968:553 }
2023-11-25 15:53:18,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:18,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1900080460=41, srv1885521601=40, srv1939313195=43, srv2081987715=50, srv1239206781=14, srv356437714=63, srv81741017=88, srv503151877=68, srv607510662=74, srv81211775=87, srv869104900=92, srv629207411=76, srv1733869849=31, srv352098288=62, srv454985016=65, srv698709161=79, srv1851483568=38, srv507068607=69, srv467156153=66, srv513664882=70, srv1302923512=17, srv907698068=93, srv1776611691=32, srv74811395=82, srv1230849139=13, srv1870268183=39, srv1993985333=45, srv971058899=96, srv1789102388=33, srv1196742950=10, srv522198109=71, srv977658306=98, srv358819637=64, srv476677145=67, srv2110472212=52, srv172878102=29, srv1840174353=37, srv1187148906=8, srv1100391391=3, srv636979114=78, srv713983562=80, srv74965688=83, srv779735741=84, srv1729870567=30, srv1053228072=2, srv1409369583=20, srv615731444=75, srv1644545990=25, srv1200857306=11, srv603363600=73, srv343892433=61, srv2046833492=49, srv635656730=77, srv2140044215=54, srv827156362=89, srv994446811=99, srv1465634968=21, srv1712939228=28, srv1180008464=7, srv146754236=22, srv2111695389=53, srv1999489407=46, srv2094413480=51, srv1809578219=34, srv264040763=58, srv1483279300=23, srv811814816=86, srv1004085584=0, srv1706631351=27, srv739583364=81, srv250666242=57, srv1126450595=4, srv1351212083=19, srv1989217915=44, srv1165233451=5, srv1937843031=42, srv247507718=56, srv1192488014=9, srv973784634=97, srv1334692077=18, srv795185943=85, srv276824228=59, srv1535563919=24, srv857381877=91, srv1007469890=1, srv1815672011=35, srv854913722=90, srv2034506011=48, srv1299465488=15, srv1702203684=26, srv1219100423=12, srv2010176502=47, srv221741840=55, srv936549343=95, srv1170766598=6, srv1816667212=36, srv304836606=60, srv566687582=72, srv920199117=94, srv1299799088=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:18,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:18,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 15:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 15:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 15:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 15:53:18,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 15:53:18,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 15:53:18,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 15:53:18,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 15:53:18,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 15:53:18,336 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00709966812198461); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01174934628169321); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00709966812198461); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:18,336 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-25 15:53:18,336 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.097748611573507E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00709966812198461); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01174934628169321); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00709966812198461); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-25 15:53:18,803 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 574 ms to try 5000 different iterations.  Found a solution that moves 274 regions; Going from a computed imbalance of 7.097748611573507E-4 to a new imbalance of 2.049603067633799E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.030444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:18,805 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1302923512:83 , srv2046833492:83 , srv304836606:83 , srv1004085584:84 , srv1100391391:84 , srv1126450595:84 , srv1165233451:84 , srv1230849139:84 , srv1239206781:84 , srv1299465488:84 , srv1299799088:84 , srv1644545990:84 , srv1702203684:84 , srv1733869849:84 , srv1776611691:84 , srv1851483568:84 , srv1900080460:84 , srv1989217915:84 , srv1999489407:84 , srv2034506011:84 , srv2110472212:84 , srv221741840:84 , srv247507718:84 , srv276824228:84 , srv358819637:84 , srv476677145:84 , srv513664882:84 , srv603363600:84 , srv629207411:84 , srv713983562:84 , srv739583364:84 , srv74811395:84 , srv795185943:84 , srv811814816:84 , srv81211775:84 , srv907698068:84 , srv971058899:84 , srv973784634:84 , srv1007469890:85 , srv1053228072:85 , srv1180008464:85 , srv1187148906:85 , srv1192488014:85 , srv1196742950:85 , srv1219100423:85 , srv1334692077:85 , srv1351212083:85 , srv1409369583:85 , srv1483279300:85 , srv1535563919:85 , srv1706631351:85 , srv1712939228:85 , srv172878102:85 , srv1729870567:85 , srv1789102388:85 , srv1809578219:85 , srv1815672011:85 , srv1816667212:85 , srv1840174353:85 , srv1870268183:85 , srv1885521601:85 , srv1937843031:85 , srv1939313195:85 , srv1993985333:85 , srv2010176502:85 , srv2081987715:85 , srv2094413480:85 , srv2111695389:85 , srv250666242:85 , srv264040763:85 , srv343892433:85 , srv352098288:85 , srv356437714:85 , srv454985016:85 , srv467156153:85 , srv503151877:85 , srv507068607:85 , srv522198109:85 , srv566687582:85 , srv607510662:85 , srv615731444:85 , srv635656730:85 , srv636979114:85 , srv698709161:85 , srv74965688:85 , srv779735741:85 , srv81741017:85 , srv827156362:85 , srv854913722:85 , srv857381877:85 , srv920199117:85 , srv936549343:85 , srv977658306:85 , srv994446811:85 , srv1170766598:86 , srv1200857306:86 , srv146754236:86 , srv2140044215:86 , srv869104900:86 , srv1465634968:621 }
2023-11-25 15:53:18,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:18,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1900080460=41, srv1885521601=40, srv1939313195=43, srv2081987715=50, srv1239206781=14, srv356437714=63, srv81741017=88, srv503151877=68, srv607510662=74, srv81211775=87, srv869104900=92, srv629207411=76, srv1733869849=31, srv352098288=62, srv454985016=65, srv698709161=79, srv1851483568=38, srv507068607=69, srv467156153=66, srv513664882=70, srv1302923512=17, srv907698068=93, srv1776611691=32, srv74811395=82, srv1230849139=13, srv1870268183=39, srv1993985333=45, srv971058899=96, srv1789102388=33, srv1196742950=10, srv522198109=71, srv977658306=98, srv358819637=64, srv476677145=67, srv2110472212=52, srv172878102=29, srv1840174353=37, srv1187148906=8, srv1100391391=3, srv636979114=78, srv713983562=80, srv74965688=83, srv779735741=84, srv1729870567=30, srv1053228072=2, srv1409369583=20, srv615731444=75, srv1644545990=25, srv1200857306=11, srv603363600=73, srv343892433=61, srv2046833492=49, srv635656730=77, srv2140044215=54, srv827156362=89, srv994446811=99, srv1465634968=21, srv1712939228=28, srv1180008464=7, srv146754236=22, srv2111695389=53, srv1999489407=46, srv2094413480=51, srv1809578219=34, srv264040763=58, srv1483279300=23, srv811814816=86, srv1004085584=0, srv1706631351=27, srv739583364=81, srv250666242=57, srv1126450595=4, srv1351212083=19, srv1989217915=44, srv1165233451=5, srv1937843031=42, srv247507718=56, srv1192488014=9, srv973784634=97, srv1334692077=18, srv795185943=85, srv276824228=59, srv1535563919=24, srv857381877=91, srv1007469890=1, srv1815672011=35, srv854913722=90, srv2034506011=48, srv1299465488=15, srv1702203684=26, srv1219100423=12, srv2010176502=47, srv221741840=55, srv936549343=95, srv1170766598=6, srv1816667212=36, srv304836606=60, srv566687582=72, srv920199117=94, srv1299799088=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:18,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:18,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-25 15:53:18,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-25 15:53:18,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-25 15:53:18,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-25 15:53:18,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-25 15:53:18,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-25 15:53:18,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-25 15:53:18,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-25 15:53:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-25 15:53:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-25 15:53:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-25 15:53:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-25 15:53:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-25 15:53:18,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8577441669512895E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:18,981 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1302923512:83 , srv2046833492:83 , srv304836606:83 , srv1004085584:84 , srv1100391391:84 , srv1126450595:84 , srv1165233451:84 , srv1230849139:84 , srv1239206781:84 , srv1299465488:84 , srv1299799088:84 , srv1644545990:84 , srv1702203684:84 , srv1733869849:84 , srv1776611691:84 , srv1851483568:84 , srv1900080460:84 , srv1989217915:84 , srv1999489407:84 , srv2034506011:84 , srv2110472212:84 , srv221741840:84 , srv247507718:84 , srv276824228:84 , srv358819637:84 , srv476677145:84 , srv513664882:84 , srv603363600:84 , srv629207411:84 , srv713983562:84 , srv739583364:84 , srv74811395:84 , srv795185943:84 , srv811814816:84 , srv81211775:84 , srv907698068:84 , srv971058899:84 , srv973784634:84 , srv1007469890:85 , srv1053228072:85 , srv1180008464:85 , srv1187148906:85 , srv1192488014:85 , srv1196742950:85 , srv1219100423:85 , srv1334692077:85 , srv1351212083:85 , srv1409369583:85 , srv1483279300:85 , srv1535563919:85 , srv1706631351:85 , srv1712939228:85 , srv172878102:85 , srv1729870567:85 , srv1789102388:85 , srv1809578219:85 , srv1815672011:85 , srv1816667212:85 , srv1840174353:85 , srv1870268183:85 , srv1885521601:85 , srv1937843031:85 , srv1939313195:85 , srv1993985333:85 , srv2010176502:85 , srv2081987715:85 , srv2094413480:85 , srv2111695389:85 , srv250666242:85 , srv264040763:85 , srv343892433:85 , srv352098288:85 , srv356437714:85 , srv454985016:85 , srv467156153:85 , srv503151877:85 , srv507068607:85 , srv522198109:85 , srv566687582:85 , srv607510662:85 , srv615731444:85 , srv635656730:85 , srv636979114:85 , srv698709161:85 , srv74965688:85 , srv779735741:85 , srv81741017:85 , srv827156362:85 , srv854913722:85 , srv857381877:85 , srv920199117:85 , srv936549343:85 , srv977658306:85 , srv994446811:85 , srv1170766598:86 , srv1200857306:86 , srv146754236:86 , srv2140044215:86 , srv869104900:86 , srv1465634968:621 }
2023-11-25 15:53:19,024 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 271), ProcessCount=172 (was 172), AvailableMemoryMB=8834 (was 9696)
2023-11-25 15:53:19,034 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=172, AvailableMemoryMB=8832
2023-11-25 15:53:19,034 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-25 15:53:19,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-25 15:53:19,050 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1184273039:84 , srv1241727291:84 , srv1248664590:84 , srv1337866246:84 , srv134497558:84 , srv1369532816:84 , srv1386954629:84 , srv1425338608:84 , srv1508643056:84 , srv1543996993:84 , srv17537917:84 , srv1844630496:84 , srv1855367207:84 , srv1877244907:84 , srv1893123130:84 , srv1930650086:84 , srv247707113:84 , srv319069920:84 , srv36208852:84 , srv483667042:84 , srv581288037:84 , srv615661706:84 , srv645842571:84 , srv658400048:84 , srv728620341:84 , srv730246113:84 , srv778379576:84 , srv93828573:84 , srv991285979:84 , srv1160998172:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-25 15:53:19,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:53:19,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv93828573=28, srv581288037=21, srv615661706=22, srv1855367207=13, srv1369532816=6, srv1844630496=12, srv1877244907=14, srv1893123130=15, srv36208852=19, srv1386954629=7, srv319069920=18, srv483667042=20, srv1508643056=9, srv247707113=17, srv1184273039=1, srv730246113=26, srv1543996993=10, srv645842571=23, srv1930650086=16, srv658400048=24, srv728620341=25, srv17537917=11, srv1248664590=3, srv1425338608=8, srv134497558=5, srv1241727291=2, srv1160998172=0, srv991285979=29, srv778379576=27, srv1337866246=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:53:19,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:53:19,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:53:19,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:53:19,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:53:19,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:53:19,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:53:19,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-25 15:53:19,092 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1625918808310587, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:53:19,093 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912705364971665, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1625918808310587, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-25 15:55:19,071 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5733074 different iterations.  Found a solution that moves 1866 regions; Going from a computed imbalance of 0.9912705364971665 to a new imbalance of 4.3553370884861656E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6911111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:55:19,076 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1160998172:90 , srv1184273039:90 , srv1241727291:90 , srv1248664590:90 , srv1337866246:90 , srv134497558:90 , srv1369532816:90 , srv1386954629:90 , srv1425338608:90 , srv1508643056:90 , srv1543996993:90 , srv17537917:90 , srv1844630496:90 , srv1855367207:90 , srv1877244907:90 , srv1893123130:90 , srv1930650086:90 , srv247707113:90 , srv319069920:90 , srv36208852:90 , srv483667042:90 , srv581288037:90 , srv615661706:90 , srv645842571:90 , srv658400048:90 , srv728620341:90 , srv730246113:90 , srv778379576:90 , srv93828573:90 , srv991285979:90 }
2023-11-25 15:55:19,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-25 15:55:19,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv93828573=28, srv581288037=21, srv615661706=22, srv1855367207=13, srv1369532816=6, srv1844630496=12, srv1877244907=14, srv1893123130=15, srv36208852=19, srv1386954629=7, srv319069920=18, srv483667042=20, srv1508643056=9, srv247707113=17, srv1184273039=1, srv730246113=26, srv1543996993=10, srv645842571=23, srv1930650086=16, srv658400048=24, srv728620341=25, srv17537917=11, srv1248664590=3, srv1425338608=8, srv134497558=5, srv1241727291=2, srv1160998172=0, srv991285979=29, srv778379576=27, srv1337866246=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-25 15:55:19,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-25 15:55:19,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-25 15:55:19,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-25 15:55:19,094 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1160998172:90 , srv1184273039:90 , srv1241727291:90 , srv1248664590:90 , srv1337866246:90 , srv134497558:90 , srv1369532816:90 , srv1386954629:90 , srv1425338608:90 , srv1508643056:90 , srv1543996993:90 , srv17537917:90 , srv1844630496:90 , srv1855367207:90 , srv1877244907:90 , srv1893123130:90 , srv1930650086:90 , srv247707113:90 , srv319069920:90 , srv36208852:90 , srv483667042:90 , srv581288037:90 , srv615661706:90 , srv645842571:90 , srv658400048:90 , srv728620341:90 , srv730246113:90 , srv778379576:90 , srv93828573:90 , srv991285979:90 }
2023-11-25 15:55:19,106 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=218 (was 271), ProcessCount=171 (was 172), AvailableMemoryMB=8185 (was 8832)