2023-07-15 10:53:12,797 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-15 10:53:13,108 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-15 10:53:13,217 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-15 10:53:13,282 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-15 10:53:13,289 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-15 10:53:13,305 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=288, ProcessCount=175, AvailableMemoryMB=7779
2023-07-15 10:53:13,307 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-15 10:53:13,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-15 10:53:13,461 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1066627173:84 , srv1080825026:84 , srv1106370938:84 , srv1111411964:84 , srv1112375893:84 , srv1115176669:84 , srv1164819153:84 , srv1174251806:84 , srv1196878172:84 , srv120506609:84 , srv1214518152:84 , srv1232966608:84 , srv1281237510:84 , srv1319125918:84 , srv1329097874:84 , srv13576172:84 , srv1385157809:84 , srv1393216441:84 , srv1422549090:84 , srv143274704:84 , srv1435722836:84 , srv1437026048:84 , srv1437945197:84 , srv1483611311:84 , srv1495668596:84 , srv150078116:84 , srv1521182093:84 , srv1539555695:84 , srv1547384389:84 , srv1561591163:84 , srv1600195911:84 , srv1612576205:84 , srv1619563913:84 , srv16397983:84 , srv1662611653:84 , srv1663657971:84 , srv1698473762:84 , srv1728801292:84 , srv1755832977:84 , srv1772954039:84 , srv1800299796:84 , srv1809511220:84 , srv1811631318:84 , srv1818458173:84 , srv1824002321:84 , srv1824971904:84 , srv1841212794:84 , srv1887171919:84 , srv1889429922:84 , srv1903449974:84 , srv1930467783:84 , srv1949049061:84 , srv1971472066:84 , srv1988448790:84 , srv2002387130:84 , srv2011359456:84 , srv2018377479:84 , srv2032284224:84 , srv2045949841:84 , srv2054988038:84 , srv208557485:84 , srv2087066350:84 , srv2111839110:84 , srv240316956:84 , srv269640248:84 , srv309177545:84 , srv3096726:84 , srv337949179:84 , srv347291657:84 , srv365071340:84 , srv367782716:84 , srv385383042:84 , srv393483794:84 , srv394643979:84 , srv401309161:84 , srv404569369:84 , srv424101095:84 , srv428772914:84 , srv452550512:84 , srv454774326:84 , srv519553247:84 , srv530516022:84 , srv547769588:84 , srv607458608:84 , srv673445339:84 , srv675139372:84 , srv68151960:84 , srv692791693:84 , srv707159928:84 , srv726044019:84 , srv728045389:84 , srv735351896:84 , srv745512620:84 , srv788476863:84 , srv849246041:84 , srv889397735:84 , srv927950148:84 , srv948684394:84 , srv987227424:84 , srv2127899489:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-15 10:53:13,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:13,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:13,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:13,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:13,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:13,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:13,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:13,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:13,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:13,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:13,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:13,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:13,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:13,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:13,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:13,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:13,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:13,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:13,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:13,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:13,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:13,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:13,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:13,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:13,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:13,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:13,827 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-15 10:53:13,827 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-15 10:53:14,163 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 532 ms to try 5000 different iterations.  Found a solution that moves 4668 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17763318456274435. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04074655605879104, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056062335035673254, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5186666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04074655605879104, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17258333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:14,178 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2111839110:79 , srv745512620:80 , srv1066627173:81 , srv120506609:82 , srv309177545:82 , srv428772914:82 , srv726044019:82 , srv1115176669:83 , srv1435722836:83 , srv1800299796:83 , srv2002387130:83 , srv452550512:83 , srv519553247:83 , srv1174251806:84 , srv1422549090:84 , srv1662611653:84 , srv1728801292:84 , srv1903449974:84 , srv2087066350:84 , srv365071340:84 , srv401309161:84 , srv530516022:84 , srv673445339:84 , srv788476863:84 , srv889397735:84 , srv1080825026:85 , srv1281237510:85 , srv1319125918:85 , srv1561591163:85 , srv1600195911:85 , srv1663657971:85 , srv1887171919:85 , srv1889429922:85 , srv2032284224:85 , srv208557485:85 , srv337949179:85 , srv385383042:85 , srv404569369:85 , srv607458608:85 , srv735351896:85 , srv1196878172:86 , srv1547384389:86 , srv1619563913:86 , srv1698473762:86 , srv1809511220:86 , srv1824002321:86 , srv240316956:86 , srv394643979:86 , srv454774326:86 , srv692791693:86 , srv927950148:86 , srv948684394:86 , srv987227424:86 , srv1393216441:87 , srv143274704:87 , srv150078116:87 , srv1539555695:87 , srv1772954039:87 , srv1811631318:87 , srv1841212794:87 , srv1971472066:87 , srv2018377479:87 , srv2045949841:87 , srv347291657:87 , srv424101095:87 , srv707159928:87 , srv728045389:87 , srv849246041:87 , srv1112375893:88 , srv1214518152:88 , srv1437945197:88 , srv1755832977:88 , srv1824971904:88 , srv1949049061:88 , srv2011359456:88 , srv2054988038:88 , srv269640248:88 , srv367782716:88 , srv393483794:88 , srv547769588:88 , srv675139372:88 , srv1164819153:89 , srv13576172:89 , srv1988448790:89 , srv3096726:89 , srv1111411964:90 , srv1385157809:90 , srv1437026048:90 , srv1495668596:90 , srv1521182093:90 , srv1329097874:91 , srv1483611311:91 , srv1930467783:91 , srv68151960:91 , srv1106370938:92 , srv1232966608:92 , srv1612576205:92 , srv16397983:92 , srv1818458173:94 , srv2127899489:452 }
2023-07-15 10:53:14,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:14,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:14,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:14,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:14,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:14,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:14,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:14,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:14,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:14,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:14,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:14,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:14,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:14,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:14,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:14,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:14,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:14,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:14,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:14,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:14,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:14,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:14,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:14,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:14,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:14,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:14,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:14,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:14,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:14,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:14,397 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04074655605879104, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056062335035673254, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04074655605879104, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17258333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:14,397 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-15 10:53:14,398 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17760049852813176, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04074655605879104, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056062335035673254, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04074655605879104, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17258333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24191666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-15 10:53:14,773 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 510 ms to try 5000 different iterations.  Found a solution that moves 2553 regions; Going from a computed imbalance of 0.17760049852813176 to a new imbalance of 0.01674668934034127. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022839900503163063); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038600663509123216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2836666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022839900503163063); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:14,791 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1612576205:78 , srv1818458173:78 , srv1232966608:79 , srv1437026048:79 , srv68151960:79 , srv1329097874:80 , srv1930467783:80 , srv1988448790:80 , srv1111411964:81 , srv1385157809:81 , srv16397983:81 , srv1495668596:82 , srv1521182093:82 , srv2054988038:82 , srv1164819153:83 , srv13576172:83 , srv1483611311:83 , srv1824971904:83 , srv2011359456:83 , srv269640248:83 , srv367782716:83 , srv1196878172:84 , srv1214518152:84 , srv1437945197:84 , srv1539555695:84 , srv1698473762:84 , srv1772954039:84 , srv2018377479:84 , srv3096726:84 , srv347291657:84 , srv393483794:84 , srv692791693:84 , srv707159928:84 , srv849246041:84 , srv948684394:84 , srv1106370938:85 , srv1112375893:85 , srv150078116:85 , srv1755832977:85 , srv2032284224:85 , srv2045949841:85 , srv2087066350:85 , srv394643979:85 , srv424101095:85 , srv547769588:85 , srv675139372:85 , srv728045389:85 , srv987227424:85 , srv1115176669:86 , srv1319125918:86 , srv143274704:86 , srv1663657971:86 , srv1809511220:86 , srv1811631318:86 , srv1841212794:86 , srv1971472066:86 , srv208557485:86 , srv240316956:86 , srv404569369:86 , srv927950148:86 , srv1281237510:87 , srv1393216441:87 , srv1422549090:87 , srv1547384389:87 , srv1728801292:87 , srv1949049061:87 , srv337949179:87 , srv530516022:87 , srv607458608:87 , srv1080825026:88 , srv1174251806:88 , srv1561591163:88 , srv1600195911:88 , srv1619563913:88 , srv1662611653:88 , srv1824002321:88 , srv1887171919:88 , srv1903449974:88 , srv385383042:88 , srv454774326:88 , srv889397735:88 , srv1066627173:89 , srv1435722836:89 , srv1800299796:89 , srv1889429922:89 , srv2002387130:89 , srv365071340:89 , srv428772914:89 , srv519553247:89 , srv788476863:89 , srv401309161:90 , srv452550512:90 , srv673445339:90 , srv735351896:90 , srv120506609:91 , srv2111839110:91 , srv726044019:91 , srv309177545:93 , srv745512620:93 , srv2127899489:525 }
2023-07-15 10:53:14,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:14,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:14,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:14,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:14,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:14,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:14,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:14,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:14,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:14,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:14,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:14,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:14,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:14,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:15,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022839900503163063); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038600663509123216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022839900503163063); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:15,026 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-15 10:53:15,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01672881285225943, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022839900503163063); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038600663509123216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022839900503163063); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-15 10:53:15,392 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 507 ms to try 5000 different iterations.  Found a solution that moves 1022 regions; Going from a computed imbalance of 0.01672881285225943 to a new imbalance of 7.812977871752903E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009847319278346619); MoveCostFunction : (multiplier=7.0, imbalance=0.11355555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:15,395 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1106370938:80 , srv1393216441:83 , srv1483611311:83 , srv1949049061:83 , srv309177545:83 , srv401309161:83 , srv454774326:83 , srv735351896:83 , srv1112375893:84 , srv1174251806:84 , srv120506609:84 , srv143274704:84 , srv1561591163:84 , srv1619563913:84 , srv1841212794:84 , srv1889429922:84 , srv365071340:84 , srv673445339:84 , srv675139372:84 , srv745512620:84 , srv788476863:84 , srv1111411964:85 , srv1164819153:85 , srv1281237510:85 , srv1435722836:85 , srv150078116:85 , srv1521182093:85 , srv1547384389:85 , srv1600195911:85 , srv1662611653:85 , srv1755832977:85 , srv1809511220:85 , srv1811631318:85 , srv1818458173:85 , srv1824002321:85 , srv1824971904:85 , srv1887171919:85 , srv1971472066:85 , srv2002387130:85 , srv3096726:85 , srv385383042:85 , srv424101095:85 , srv452550512:85 , srv519553247:85 , srv547769588:85 , srv726044019:85 , srv728045389:85 , srv927950148:85 , srv987227424:85 , srv1066627173:86 , srv1080825026:86 , srv1214518152:86 , srv1232966608:86 , srv1319125918:86 , srv1329097874:86 , srv13576172:86 , srv1385157809:86 , srv1422549090:86 , srv1437945197:86 , srv1495668596:86 , srv1539555695:86 , srv16397983:86 , srv1728801292:86 , srv1772954039:86 , srv1800299796:86 , srv1903449974:86 , srv2011359456:86 , srv2018377479:86 , srv2045949841:86 , srv2111839110:86 , srv240316956:86 , srv347291657:86 , srv367782716:86 , srv393483794:86 , srv394643979:86 , srv428772914:86 , srv530516022:86 , srv607458608:86 , srv707159928:86 , srv849246041:86 , srv889397735:86 , srv1612576205:87 , srv1663657971:87 , srv1698473762:87 , srv1930467783:87 , srv2032284224:87 , srv208557485:87 , srv269640248:87 , srv337949179:87 , srv68151960:87 , srv692791693:87 , srv948684394:87 , srv1115176669:88 , srv1196878172:88 , srv1988448790:88 , srv2054988038:88 , srv404569369:88 , srv1437026048:89 , srv2087066350:89 , srv2127899489:540 }
2023-07-15 10:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:15,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:15,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:15,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:15,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:15,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:15,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:15,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:15,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:15,545 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009847319278346619); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:15,546 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-15 10:53:15,546 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.741415902228258E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0066647958467430145); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009847319278346619); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0066647958467430145); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-15 10:53:15,921 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 266 regions; Going from a computed imbalance of 7.741415902228258E-4 to a new imbalance of 7.391961557089548E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.029555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:15,922 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1080825026:83 , srv1663657971:83 , srv1824002321:83 , srv1949049061:83 , srv3096726:83 , srv337949179:83 , srv404569369:83 , srv452550512:83 , srv1196878172:84 , srv1214518152:84 , srv1232966608:84 , srv13576172:84 , srv1437026048:84 , srv1437945197:84 , srv1495668596:84 , srv150078116:84 , srv1547384389:84 , srv1600195911:84 , srv1612576205:84 , srv1619563913:84 , srv16397983:84 , srv1698473762:84 , srv1755832977:84 , srv1800299796:84 , srv1818458173:84 , srv1887171919:84 , srv1889429922:84 , srv1930467783:84 , srv1971472066:84 , srv2054988038:84 , srv208557485:84 , srv2087066350:84 , srv240316956:84 , srv269640248:84 , srv309177545:84 , srv393483794:84 , srv394643979:84 , srv530516022:84 , srv547769588:84 , srv673445339:84 , srv68151960:84 , srv726044019:84 , srv728045389:84 , srv735351896:84 , srv889397735:84 , srv1106370938:85 , srv1115176669:85 , srv1164819153:85 , srv120506609:85 , srv1281237510:85 , srv1319125918:85 , srv1329097874:85 , srv1422549090:85 , srv143274704:85 , srv1483611311:85 , srv1521182093:85 , srv1539555695:85 , srv1561591163:85 , srv1728801292:85 , srv1772954039:85 , srv1811631318:85 , srv1841212794:85 , srv1903449974:85 , srv2002387130:85 , srv2011359456:85 , srv2018377479:85 , srv2032284224:85 , srv2045949841:85 , srv347291657:85 , srv365071340:85 , srv367782716:85 , srv385383042:85 , srv401309161:85 , srv424101095:85 , srv428772914:85 , srv454774326:85 , srv519553247:85 , srv607458608:85 , srv675139372:85 , srv692791693:85 , srv707159928:85 , srv745512620:85 , srv788476863:85 , srv849246041:85 , srv927950148:85 , srv948684394:85 , srv1066627173:86 , srv1111411964:86 , srv1112375893:86 , srv1174251806:86 , srv1385157809:86 , srv1393216441:86 , srv1435722836:86 , srv1662611653:86 , srv1809511220:86 , srv1824971904:86 , srv1988448790:86 , srv2111839110:86 , srv987227424:86 , srv2127899489:625 }
2023-07-15 10:53:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:15,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:15,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:15,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:15,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:15,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:15,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:15,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:15,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:15,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:15,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:15,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:15,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:15,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:15,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:15,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:15,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:15,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:15,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:16,073 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:16,074 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-15 10:53:16,074 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.205704376134995E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-15 10:53:16,351 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 375 ms to try 5000 different iterations.  Found a solution that moves 48 regions; Going from a computed imbalance of 7.205704376134995E-5 to a new imbalance of 7.409342480006837E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.005333333333333333); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:16,352 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1112375893:83 , srv1385157809:83 , srv1393216441:83 , srv1435722836:83 , srv16397983:83 , srv1662611653:83 , srv1809511220:83 , srv1811631318:83 , srv1903449974:83 , srv1988448790:83 , srv2045949841:83 , srv385383042:83 , srv607458608:83 , srv1066627173:84 , srv1080825026:84 , srv1106370938:84 , srv1111411964:84 , srv1115176669:84 , srv1164819153:84 , srv1174251806:84 , srv1196878172:84 , srv120506609:84 , srv1214518152:84 , srv1281237510:84 , srv1319125918:84 , srv1329097874:84 , srv13576172:84 , srv1422549090:84 , srv143274704:84 , srv1437026048:84 , srv1437945197:84 , srv1483611311:84 , srv1495668596:84 , srv1521182093:84 , srv1539555695:84 , srv1547384389:84 , srv1561591163:84 , srv1600195911:84 , srv1619563913:84 , srv1728801292:84 , srv1755832977:84 , srv1772954039:84 , srv1800299796:84 , srv1824002321:84 , srv1824971904:84 , srv1841212794:84 , srv1887171919:84 , srv1889429922:84 , srv1930467783:84 , srv1971472066:84 , srv2002387130:84 , srv2011359456:84 , srv2018377479:84 , srv2032284224:84 , srv2054988038:84 , srv208557485:84 , srv2087066350:84 , srv2111839110:84 , srv240316956:84 , srv269640248:84 , srv309177545:84 , srv337949179:84 , srv347291657:84 , srv365071340:84 , srv367782716:84 , srv393483794:84 , srv401309161:84 , srv404569369:84 , srv424101095:84 , srv428772914:84 , srv454774326:84 , srv519553247:84 , srv547769588:84 , srv673445339:84 , srv675139372:84 , srv692791693:84 , srv707159928:84 , srv726044019:84 , srv735351896:84 , srv745512620:84 , srv788476863:84 , srv849246041:84 , srv889397735:84 , srv927950148:84 , srv948684394:84 , srv987227424:84 , srv1232966608:85 , srv150078116:85 , srv1612576205:85 , srv1663657971:85 , srv1698473762:85 , srv1818458173:85 , srv1949049061:85 , srv3096726:85 , srv394643979:85 , srv452550512:85 , srv530516022:85 , srv68151960:85 , srv728045389:85 , srv2127899489:684 }
2023-07-15 10:53:16,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:16,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1319125918=13, srv240316956=64, srv2087066350=61, srv547769588=83, srv1329097874=14, srv365071340=70, srv1437026048=21, srv2018377479=56, srv1619563913=32, srv1106370938=2, srv1811631318=42, srv673445339=85, srv1232966608=11, srv1196878172=8, srv735351896=92, srv745512620=93, srv3096726=67, srv1115176669=5, srv309177545=66, srv1772954039=39, srv607458608=84, srv2011359456=55, srv530516022=82, srv1755832977=38, srv2111839110=62, srv143274704=19, srv692791693=88, srv1547384389=28, srv68151960=87, srv889397735=96, srv347291657=69, srv1495668596=24, srv1887171919=47, srv120506609=9, srv1435722836=20, srv1600195911=30, srv1393216441=17, srv1066627173=0, srv394643979=74, srv1111411964=3, srv849246041=95, srv728045389=91, srv1698473762=36, srv2002387130=54, srv948684394=98, srv1539555695=27, srv519553247=81, srv1483611311=23, srv404569369=76, srv927950148=97, srv1818458173=43, srv401309161=75, srv1080825026=1, srv1809511220=41, srv1561591163=29, srv424101095=77, srv788476863=94, srv1662611653=34, srv1281237510=12, srv675139372=86, srv13576172=15, srv1422549090=18, srv726044019=90, srv385383042=72, srv454774326=80, srv2045949841=58, srv987227424=99, srv1612576205=31, srv1174251806=7, srv1521182093=26, srv1841212794=46, srv1824002321=44, srv150078116=25, srv393483794=73, srv1663657971=35, srv2054988038=59, srv367782716=71, srv1949049061=51, srv208557485=60, srv337949179=68, srv1800299796=40, srv1971472066=52, srv1214518152=10, srv1385157809=16, srv1988448790=53, srv2127899489=63, srv1728801292=37, srv1930467783=50, srv269640248=65, srv1112375893=4, srv428772914=78, srv1824971904=45, srv1903449974=49, srv1164819153=6, srv452550512=79, srv1437945197=22, srv707159928=89, srv16397983=33, srv1889429922=48, srv2032284224=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:16,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:16,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-15 10:53:16,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-15 10:53:16,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-15 10:53:16,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-15 10:53:16,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-15 10:53:16,479 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:16,480 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1112375893:83 , srv1385157809:83 , srv1393216441:83 , srv1435722836:83 , srv16397983:83 , srv1662611653:83 , srv1809511220:83 , srv1811631318:83 , srv1903449974:83 , srv1988448790:83 , srv2045949841:83 , srv385383042:83 , srv607458608:83 , srv1066627173:84 , srv1080825026:84 , srv1106370938:84 , srv1111411964:84 , srv1115176669:84 , srv1164819153:84 , srv1174251806:84 , srv1196878172:84 , srv120506609:84 , srv1214518152:84 , srv1281237510:84 , srv1319125918:84 , srv1329097874:84 , srv13576172:84 , srv1422549090:84 , srv143274704:84 , srv1437026048:84 , srv1437945197:84 , srv1483611311:84 , srv1495668596:84 , srv1521182093:84 , srv1539555695:84 , srv1547384389:84 , srv1561591163:84 , srv1600195911:84 , srv1619563913:84 , srv1728801292:84 , srv1755832977:84 , srv1772954039:84 , srv1800299796:84 , srv1824002321:84 , srv1824971904:84 , srv1841212794:84 , srv1887171919:84 , srv1889429922:84 , srv1930467783:84 , srv1971472066:84 , srv2002387130:84 , srv2011359456:84 , srv2018377479:84 , srv2032284224:84 , srv2054988038:84 , srv208557485:84 , srv2087066350:84 , srv2111839110:84 , srv240316956:84 , srv269640248:84 , srv309177545:84 , srv337949179:84 , srv347291657:84 , srv365071340:84 , srv367782716:84 , srv393483794:84 , srv401309161:84 , srv404569369:84 , srv424101095:84 , srv428772914:84 , srv454774326:84 , srv519553247:84 , srv547769588:84 , srv673445339:84 , srv675139372:84 , srv692791693:84 , srv707159928:84 , srv726044019:84 , srv735351896:84 , srv745512620:84 , srv788476863:84 , srv849246041:84 , srv889397735:84 , srv927950148:84 , srv948684394:84 , srv987227424:84 , srv1232966608:85 , srv150078116:85 , srv1612576205:85 , srv1663657971:85 , srv1698473762:85 , srv1818458173:85 , srv1949049061:85 , srv3096726:85 , srv394643979:85 , srv452550512:85 , srv530516022:85 , srv68151960:85 , srv728045389:85 , srv2127899489:684 }
2023-07-15 10:53:16,507 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=313 (was 288) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=7052 (was 7779)
2023-07-15 10:53:16,517 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=175, AvailableMemoryMB=7052
2023-07-15 10:53:16,517 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-15 10:53:16,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-15 10:53:16,530 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1105574748:84 , srv1123149957:84 , srv1412221984:84 , srv1426952859:84 , srv1459653172:84 , srv1588520551:84 , srv1627527598:84 , srv1652689646:84 , srv1667665078:84 , srv1675273026:84 , srv1773831544:84 , srv1775457413:84 , srv1788270400:84 , srv1797005848:84 , srv1870290012:84 , srv1871270842:84 , srv1881254097:84 , srv1951193815:84 , srv1987900523:84 , srv2071946547:84 , srv2119590629:84 , srv370753075:84 , srv494161956:84 , srv541311033:84 , srv624393740:84 , srv654462583:84 , srv699488580:84 , srv852899848:84 , srv928625128:84 , srv1196754953:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-15 10:53:16,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:53:16,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1773831544=11, srv370753075=22, srv1797005848=14, srv624393740=25, srv1871270842=16, srv1123149957=1, srv1775457413=12, srv1412221984=3, srv1426952859=4, srv1652689646=8, srv1667665078=9, srv928625128=29, srv1788270400=13, srv2119590629=21, srv1459653172=5, srv2071946547=20, srv1951193815=18, srv1588520551=6, srv494161956=23, srv1987900523=19, srv1870290012=15, srv654462583=26, srv1881254097=17, srv1627527598=7, srv1196754953=2, srv541311033=24, srv699488580=27, srv852899848=28, srv1105574748=0, srv1675273026=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:53:16,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:53:16,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:53:16,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-15 10:53:16,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1237859342830518, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:53:16,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912583088701223, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1237859342830518, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-15 10:55:16,551 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5427771 different iterations.  Found a solution that moves 1869 regions; Going from a computed imbalance of 0.9912583088701223 to a new imbalance of 4.362339238146111E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:55:16,557 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1105574748:90 , srv1123149957:90 , srv1196754953:90 , srv1412221984:90 , srv1426952859:90 , srv1459653172:90 , srv1588520551:90 , srv1627527598:90 , srv1652689646:90 , srv1667665078:90 , srv1675273026:90 , srv1773831544:90 , srv1775457413:90 , srv1788270400:90 , srv1797005848:90 , srv1870290012:90 , srv1871270842:90 , srv1881254097:90 , srv1951193815:90 , srv1987900523:90 , srv2071946547:90 , srv2119590629:90 , srv370753075:90 , srv494161956:90 , srv541311033:90 , srv624393740:90 , srv654462583:90 , srv699488580:90 , srv852899848:90 , srv928625128:90 }
2023-07-15 10:55:16,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-15 10:55:16,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1773831544=11, srv370753075=22, srv1797005848=14, srv624393740=25, srv1871270842=16, srv1123149957=1, srv1775457413=12, srv1412221984=3, srv1426952859=4, srv1652689646=8, srv1667665078=9, srv928625128=29, srv1788270400=13, srv2119590629=21, srv1459653172=5, srv2071946547=20, srv1951193815=18, srv1588520551=6, srv494161956=23, srv1987900523=19, srv1870290012=15, srv654462583=26, srv1881254097=17, srv1627527598=7, srv1196754953=2, srv541311033=24, srv699488580=27, srv852899848=28, srv1105574748=0, srv1675273026=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-15 10:55:16,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-15 10:55:16,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-15 10:55:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-15 10:55:16,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-15 10:55:16,575 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1105574748:90 , srv1123149957:90 , srv1196754953:90 , srv1412221984:90 , srv1426952859:90 , srv1459653172:90 , srv1588520551:90 , srv1627527598:90 , srv1652689646:90 , srv1667665078:90 , srv1675273026:90 , srv1773831544:90 , srv1775457413:90 , srv1788270400:90 , srv1797005848:90 , srv1870290012:90 , srv1871270842:90 , srv1881254097:90 , srv1951193815:90 , srv1987900523:90 , srv2071946547:90 , srv2119590629:90 , srv370753075:90 , srv494161956:90 , srv541311033:90 , srv624393740:90 , srv654462583:90 , srv699488580:90 , srv852899848:90 , srv928625128:90 }
2023-07-15 10:55:16,587 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 313), ProcessCount=176 (was 175) - ProcessCount LEAK? -, AvailableMemoryMB=6439 (was 7052)