2023-08-16 08:53:13,035 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-16 08:53:13,371 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-16 08:53:13,487 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-16 08:53:13,549 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 08:53:13,556 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 08:53:13,570 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=174, AvailableMemoryMB=2456
2023-08-16 08:53:13,572 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 08:53:13,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 08:53:13,724 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000267794:84 , srv1005788168:84 , srv1008176801:84 , srv1059128245:84 , srv1100264458:84 , srv1101290040:84 , srv110259878:84 , srv1102734357:84 , srv1169861291:84 , srv1171409312:84 , srv1172761908:84 , srv1203945641:84 , srv1204419426:84 , srv121638701:84 , srv1280766628:84 , srv1287572336:84 , srv1300974439:84 , srv1314270655:84 , srv1419616974:84 , srv1428240504:84 , srv1432389981:84 , srv1434580885:84 , srv1440223321:84 , srv1471561231:84 , srv1504018709:84 , srv1512403812:84 , srv1592752585:84 , srv1594594124:84 , srv1612481727:84 , srv1630696858:84 , srv1632286796:84 , srv1642756771:84 , srv1648566935:84 , srv1726814018:84 , srv1744573146:84 , srv1783322089:84 , srv1827079851:84 , srv1845570786:84 , srv1899502598:84 , srv2015432744:84 , srv2031172082:84 , srv2046811328:84 , srv2048950486:84 , srv2053899869:84 , srv2072954566:84 , srv2092471043:84 , srv211053420:84 , srv2116210204:84 , srv214245181:84 , srv219644809:84 , srv227744701:84 , srv238421735:84 , srv246416164:84 , srv28670266:84 , srv310016121:84 , srv340671675:84 , srv353795964:84 , srv356672087:84 , srv37174641:84 , srv379281828:84 , srv382373848:84 , srv411464922:84 , srv427202821:84 , srv449286063:84 , srv473078126:84 , srv498916979:84 , srv553121678:84 , srv556335468:84 , srv570593514:84 , srv609887445:84 , srv624985780:84 , srv62523307:84 , srv650932420:84 , srv667504155:84 , srv679238944:84 , srv682968112:84 , srv694156289:84 , srv7040791:84 , srv72059144:84 , srv722009939:84 , srv733503063:84 , srv748729315:84 , srv763479044:84 , srv806733951:84 , srv819011036:84 , srv822256896:84 , srv824656415:84 , srv847158180:84 , srv86796673:84 , srv869696087:84 , srv876331554:84 , srv8915991:84 , srv906930057:84 , srv908270480:84 , srv915516982:84 , srv916622209:84 , srv924570053:84 , srv967723505:84 , srv996076198:84 , srv536234229:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-16 08:53:13,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:13,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv427202821=62, srv967723505=98, srv570593514=69, srv2072954566=44, srv806733951=84, srv536234229=66, srv1059128245=3, srv2031172082=40, srv667504155=74, srv819011036=85, srv1287572336=15, srv37174641=58, srv498916979=65, srv1432389981=20, srv86796673=89, srv7040791=78, srv411464922=61, srv1648566935=32, srv382373848=60, srv121638701=13, srv908270480=94, srv722009939=80, srv2092471043=45, srv924570053=97, srv379281828=59, srv1419616974=18, srv553121678=67, srv1169861291=8, srv822256896=86, srv1005788168=1, srv1314270655=17, srv1172761908=10, srv679238944=75, srv1845570786=37, srv996076198=99, srv1512403812=25, srv1471561231=23, srv1008176801=2, srv246416164=52, srv915516982=95, srv1504018709=24, srv1630696858=29, srv1632286796=30, srv733503063=81, srv1102734357=7, srv356672087=57, srv682968112=76, srv847158180=88, srv1594594124=27, srv650932420=73, srv1592752585=26, srv1827079851=36, srv1101290040=5, srv1300974439=16, srv624985780=71, srv1203945641=11, srv62523307=72, srv1783322089=35, srv763479044=83, srv1000267794=0, srv8915991=92, srv1280766628=14, srv1171409312=9, srv1204419426=12, srv310016121=54, srv694156289=77, srv609887445=70, srv1434580885=21, srv1726814018=33, srv876331554=91, srv2116210204=47, srv869696087=90, srv748729315=82, srv1899502598=38, srv556335468=68, srv110259878=6, srv906930057=93, srv353795964=56, srv473078126=64, srv340671675=55, srv28670266=53, srv214245181=48, srv219644809=49, srv227744701=50, srv1428240504=19, srv1440223321=22, srv1642756771=31, srv824656415=87, srv2015432744=39, srv1744573146=34, srv211053420=46, srv2046811328=41, srv72059144=79, srv2053899869=43, srv1100264458=4, srv238421735=51, srv916622209=96, srv1612481727=28, srv2048950486=42, srv449286063=63} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:13,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:13,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:13,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 08:53:13,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 08:53:13,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 08:53:13,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 08:53:13,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 08:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 08:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 08:53:13,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 08:53:13,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 08:53:14,044 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:14,045 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 08:53:14,045 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 08:53:14,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 556 ms to try 5000 different iterations.  Found a solution that moves 4687 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17909270768719576. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0368094014677435, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04948443508191219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5207777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0368094014677435, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24033333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:14,446 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2046811328:82 , srv1005788168:83 , srv1428240504:83 , srv1512403812:83 , srv1592752585:83 , srv1594594124:83 , srv1726814018:83 , srv1744573146:83 , srv1827079851:83 , srv411464922:83 , srv1314270655:84 , srv1432389981:84 , srv1440223321:84 , srv1648566935:84 , srv28670266:84 , srv382373848:84 , srv427202821:84 , srv667504155:84 , srv679238944:84 , srv847158180:84 , srv996076198:84 , srv1102734357:85 , srv1169861291:85 , srv2031172082:85 , srv219644809:85 , srv62523307:85 , srv7040791:85 , srv908270480:85 , srv110259878:86 , srv121638701:86 , srv1419616974:86 , srv1630696858:86 , srv1632286796:86 , srv2072954566:86 , srv214245181:86 , srv449286063:86 , srv498916979:86 , srv553121678:86 , srv650932420:86 , srv682968112:86 , srv694156289:86 , srv72059144:86 , srv722009939:86 , srv733503063:86 , srv824656415:86 , srv86796673:86 , srv869696087:86 , srv906930057:86 , srv924570053:86 , srv1059128245:87 , srv1100264458:87 , srv1172761908:87 , srv1203945641:87 , srv1280766628:87 , srv1471561231:87 , srv1642756771:87 , srv1783322089:87 , srv1845570786:87 , srv1899502598:87 , srv2048950486:87 , srv2053899869:87 , srv2092471043:87 , srv211053420:87 , srv246416164:87 , srv340671675:87 , srv356672087:87 , srv379281828:87 , srv473078126:87 , srv556335468:87 , srv609887445:87 , srv624985780:87 , srv748729315:87 , srv822256896:87 , srv1000267794:88 , srv1504018709:88 , srv238421735:88 , srv353795964:88 , srv763479044:88 , srv806733951:88 , srv915516982:88 , srv1171409312:89 , srv2116210204:89 , srv819011036:89 , srv876331554:89 , srv8915991:89 , srv916622209:89 , srv967723505:89 , srv2015432744:90 , srv227744701:90 , srv1204419426:91 , srv37174641:91 , srv1101290040:92 , srv1300974439:92 , srv1434580885:92 , srv1612481727:92 , srv310016121:92 , srv570593514:92 , srv1287572336:93 , srv1008176801:95 , srv536234229:417 }
2023-08-16 08:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv427202821=62, srv967723505=98, srv570593514=69, srv2072954566=44, srv806733951=84, srv536234229=66, srv1059128245=3, srv2031172082=40, srv667504155=74, srv819011036=85, srv1287572336=15, srv37174641=58, srv498916979=65, srv1432389981=20, srv86796673=89, srv7040791=78, srv411464922=61, srv1648566935=32, srv382373848=60, srv121638701=13, srv908270480=94, srv722009939=80, srv2092471043=45, srv924570053=97, srv379281828=59, srv1419616974=18, srv553121678=67, srv1169861291=8, srv822256896=86, srv1005788168=1, srv1314270655=17, srv1172761908=10, srv679238944=75, srv1845570786=37, srv996076198=99, srv1512403812=25, srv1471561231=23, srv1008176801=2, srv246416164=52, srv915516982=95, srv1504018709=24, srv1630696858=29, srv1632286796=30, srv733503063=81, srv1102734357=7, srv356672087=57, srv682968112=76, srv847158180=88, srv1594594124=27, srv650932420=73, srv1592752585=26, srv1827079851=36, srv1101290040=5, srv1300974439=16, srv624985780=71, srv1203945641=11, srv62523307=72, srv1783322089=35, srv763479044=83, srv1000267794=0, srv8915991=92, srv1280766628=14, srv1171409312=9, srv1204419426=12, srv310016121=54, srv694156289=77, srv609887445=70, srv1434580885=21, srv1726814018=33, srv876331554=91, srv2116210204=47, srv869696087=90, srv748729315=82, srv1899502598=38, srv556335468=68, srv110259878=6, srv906930057=93, srv353795964=56, srv473078126=64, srv340671675=55, srv28670266=53, srv214245181=48, srv219644809=49, srv227744701=50, srv1428240504=19, srv1440223321=22, srv1642756771=31, srv824656415=87, srv2015432744=39, srv1744573146=34, srv211053420=46, srv2046811328=41, srv72059144=79, srv2053899869=43, srv1100264458=4, srv238421735=51, srv916622209=96, srv1612481727=28, srv2048950486=42, srv449286063=63} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:14,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:14,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:14,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 08:53:14,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 08:53:14,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 08:53:14,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 08:53:14,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 08:53:14,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 08:53:14,604 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0368094014677435, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04948443508191219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0368094014677435, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24033333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:14,604 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 08:53:14,605 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1790598886117396, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0368094014677435, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04948443508191219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0368094014677435, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24033333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 08:53:14,954 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 2650 regions; Going from a computed imbalance of 0.1790598886117396 to a new imbalance of 0.014831204120871598. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019586064355741565); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024732009185243015); MoveCostFunction : (multiplier=7.0, imbalance=0.29444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019586064355741565); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:14,970 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1008176801:78 , srv1287572336:78 , srv1101290040:79 , srv1204419426:80 , srv1300974439:80 , srv1612481727:81 , srv310016121:81 , srv1000267794:82 , srv1504018709:82 , srv2015432744:82 , srv2116210204:82 , srv570593514:82 , srv876331554:82 , srv967723505:82 , srv1059128245:83 , srv227744701:83 , srv353795964:83 , srv37174641:83 , srv556335468:83 , srv624985780:83 , srv763479044:83 , srv806733951:83 , srv819011036:83 , srv1171409312:84 , srv1899502598:84 , srv2053899869:84 , srv356672087:84 , srv650932420:84 , srv8915991:84 , srv906930057:84 , srv915516982:84 , srv916622209:84 , srv1419616974:85 , srv1434580885:85 , srv1630696858:85 , srv2048950486:85 , srv211053420:85 , srv238421735:85 , srv246416164:85 , srv553121678:85 , srv609887445:85 , srv733503063:85 , srv822256896:85 , srv869696087:85 , srv1100264458:86 , srv1172761908:86 , srv121638701:86 , srv1845570786:86 , srv2031172082:86 , srv2072954566:86 , srv2092471043:86 , srv214245181:86 , srv219644809:86 , srv340671675:86 , srv379281828:86 , srv449286063:86 , srv473078126:86 , srv824656415:86 , srv847158180:86 , srv86796673:86 , srv924570053:86 , srv1102734357:87 , srv1169861291:87 , srv1203945641:87 , srv1280766628:87 , srv1471561231:87 , srv1632286796:87 , srv1642756771:87 , srv1648566935:87 , srv1726814018:87 , srv1783322089:87 , srv682968112:87 , srv722009939:87 , srv748729315:87 , srv1005788168:88 , srv110259878:88 , srv1428240504:88 , srv1432389981:88 , srv1440223321:88 , srv1744573146:88 , srv1827079851:88 , srv28670266:88 , srv498916979:88 , srv62523307:88 , srv679238944:88 , srv694156289:88 , srv72059144:88 , srv908270480:88 , srv1314270655:89 , srv1592752585:89 , srv667504155:89 , srv7040791:89 , srv996076198:89 , srv382373848:90 , srv411464922:90 , srv427202821:90 , srv1512403812:91 , srv1594594124:92 , srv2046811328:92 , srv536234229:531 }
2023-08-16 08:53:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv427202821=62, srv967723505=98, srv570593514=69, srv2072954566=44, srv806733951=84, srv536234229=66, srv1059128245=3, srv2031172082=40, srv667504155=74, srv819011036=85, srv1287572336=15, srv37174641=58, srv498916979=65, srv1432389981=20, srv86796673=89, srv7040791=78, srv411464922=61, srv1648566935=32, srv382373848=60, srv121638701=13, srv908270480=94, srv722009939=80, srv2092471043=45, srv924570053=97, srv379281828=59, srv1419616974=18, srv553121678=67, srv1169861291=8, srv822256896=86, srv1005788168=1, srv1314270655=17, srv1172761908=10, srv679238944=75, srv1845570786=37, srv996076198=99, srv1512403812=25, srv1471561231=23, srv1008176801=2, srv246416164=52, srv915516982=95, srv1504018709=24, srv1630696858=29, srv1632286796=30, srv733503063=81, srv1102734357=7, srv356672087=57, srv682968112=76, srv847158180=88, srv1594594124=27, srv650932420=73, srv1592752585=26, srv1827079851=36, srv1101290040=5, srv1300974439=16, srv624985780=71, srv1203945641=11, srv62523307=72, srv1783322089=35, srv763479044=83, srv1000267794=0, srv8915991=92, srv1280766628=14, srv1171409312=9, srv1204419426=12, srv310016121=54, srv694156289=77, srv609887445=70, srv1434580885=21, srv1726814018=33, srv876331554=91, srv2116210204=47, srv869696087=90, srv748729315=82, srv1899502598=38, srv556335468=68, srv110259878=6, srv906930057=93, srv353795964=56, srv473078126=64, srv340671675=55, srv28670266=53, srv214245181=48, srv219644809=49, srv227744701=50, srv1428240504=19, srv1440223321=22, srv1642756771=31, srv824656415=87, srv2015432744=39, srv1744573146=34, srv211053420=46, srv2046811328=41, srv72059144=79, srv2053899869=43, srv1100264458=4, srv238421735=51, srv916622209=96, srv1612481727=28, srv2048950486=42, srv449286063=63} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:14,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:14,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 08:53:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 08:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 08:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 08:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 08:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 08:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 08:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 08:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 08:53:15,134 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019586064355741565); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024732009185243015); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019586064355741565); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:15,134 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 08:53:15,134 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014812648424272741, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019586064355741565); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024732009185243015); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019586064355741565); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 08:53:15,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 494 ms to try 5000 different iterations.  Found a solution that moves 928 regions; Going from a computed imbalance of 0.014812648424272741 to a new imbalance of 9.237523768467971E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005102759975884884); MoveCostFunction : (multiplier=7.0, imbalance=0.10311111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0055); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:15,518 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1434580885:82 , srv110259878:83 , srv1203945641:83 , srv1280766628:83 , srv1512403812:83 , srv1594594124:83 , srv1783322089:83 , srv2046811328:83 , srv379281828:83 , srv382373848:83 , srv427202821:83 , srv498916979:83 , srv1471561231:84 , srv1632286796:84 , srv1642756771:84 , srv227744701:84 , srv37174641:84 , srv570593514:84 , srv62523307:84 , srv682968112:84 , srv694156289:84 , srv7040791:84 , srv72059144:84 , srv748729315:84 , srv996076198:84 , srv1008176801:85 , srv1100264458:85 , srv1102734357:85 , srv1171409312:85 , srv1172761908:85 , srv1300974439:85 , srv1440223321:85 , srv1845570786:85 , srv2015432744:85 , srv2048950486:85 , srv211053420:85 , srv238421735:85 , srv310016121:85 , srv340671675:85 , srv411464922:85 , srv449286063:85 , srv473078126:85 , srv609887445:85 , srv667504155:85 , srv679238944:85 , srv722009939:85 , srv819011036:85 , srv822256896:85 , srv824656415:85 , srv86796673:85 , srv8915991:85 , srv915516982:85 , srv916622209:85 , srv1005788168:86 , srv1169861291:86 , srv1204419426:86 , srv121638701:86 , srv1287572336:86 , srv1314270655:86 , srv1419616974:86 , srv1428240504:86 , srv1592752585:86 , srv1612481727:86 , srv1630696858:86 , srv1648566935:86 , srv1899502598:86 , srv2053899869:86 , srv2072954566:86 , srv2092471043:86 , srv2116210204:86 , srv214245181:86 , srv246416164:86 , srv28670266:86 , srv553121678:86 , srv733503063:86 , srv763479044:86 , srv876331554:86 , srv908270480:86 , srv924570053:86 , srv1000267794:87 , srv1101290040:87 , srv1432389981:87 , srv1504018709:87 , srv1726814018:87 , srv1744573146:87 , srv1827079851:87 , srv2031172082:87 , srv219644809:87 , srv353795964:87 , srv356672087:87 , srv624985780:87 , srv650932420:87 , srv806733951:87 , srv847158180:87 , srv869696087:87 , srv967723505:87 , srv1059128245:88 , srv556335468:88 , srv906930057:88 , srv536234229:554 }
2023-08-16 08:53:15,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:15,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv427202821=62, srv967723505=98, srv570593514=69, srv2072954566=44, srv806733951=84, srv536234229=66, srv1059128245=3, srv2031172082=40, srv667504155=74, srv819011036=85, srv1287572336=15, srv37174641=58, srv498916979=65, srv1432389981=20, srv86796673=89, srv7040791=78, srv411464922=61, srv1648566935=32, srv382373848=60, srv121638701=13, srv908270480=94, srv722009939=80, srv2092471043=45, srv924570053=97, srv379281828=59, srv1419616974=18, srv553121678=67, srv1169861291=8, srv822256896=86, srv1005788168=1, srv1314270655=17, srv1172761908=10, srv679238944=75, srv1845570786=37, srv996076198=99, srv1512403812=25, srv1471561231=23, srv1008176801=2, srv246416164=52, srv915516982=95, srv1504018709=24, srv1630696858=29, srv1632286796=30, srv733503063=81, srv1102734357=7, srv356672087=57, srv682968112=76, srv847158180=88, srv1594594124=27, srv650932420=73, srv1592752585=26, srv1827079851=36, srv1101290040=5, srv1300974439=16, srv624985780=71, srv1203945641=11, srv62523307=72, srv1783322089=35, srv763479044=83, srv1000267794=0, srv8915991=92, srv1280766628=14, srv1171409312=9, srv1204419426=12, srv310016121=54, srv694156289=77, srv609887445=70, srv1434580885=21, srv1726814018=33, srv876331554=91, srv2116210204=47, srv869696087=90, srv748729315=82, srv1899502598=38, srv556335468=68, srv110259878=6, srv906930057=93, srv353795964=56, srv473078126=64, srv340671675=55, srv28670266=53, srv214245181=48, srv219644809=49, srv227744701=50, srv1428240504=19, srv1440223321=22, srv1642756771=31, srv824656415=87, srv2015432744=39, srv1744573146=34, srv211053420=46, srv2046811328=41, srv72059144=79, srv2053899869=43, srv1100264458=4, srv238421735=51, srv916622209=96, srv1612481727=28, srv2048950486=42, srv449286063=63} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:15,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:15,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:15,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 08:53:15,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 08:53:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 08:53:15,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 08:53:15,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 08:53:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 08:53:15,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 08:53:15,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 08:53:15,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005102759975884884); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0055); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:15,687 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 08:53:15,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.172543819623675E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005102759975884884); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0055); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 08:53:16,033 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 465 ms to try 5000 different iterations.  Found a solution that moves 230 regions; Going from a computed imbalance of 9.172543819623675E-4 to a new imbalance of 1.9374435969022173E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.025555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:16,034 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1432389981:82 , srv1434580885:83 , srv1612481727:83 , srv2092471043:83 , srv667504155:83 , srv908270480:83 , srv1008176801:84 , srv1059128245:84 , srv1100264458:84 , srv1169861291:84 , srv1171409312:84 , srv1172761908:84 , srv1204419426:84 , srv1287572336:84 , srv1314270655:84 , srv1419616974:84 , srv1471561231:84 , srv1592752585:84 , srv1594594124:84 , srv1632286796:84 , srv1642756771:84 , srv1744573146:84 , srv1827079851:84 , srv1845570786:84 , srv2031172082:84 , srv2072954566:84 , srv211053420:84 , srv214245181:84 , srv246416164:84 , srv28670266:84 , srv310016121:84 , srv340671675:84 , srv353795964:84 , srv356672087:84 , srv37174641:84 , srv411464922:84 , srv473078126:84 , srv556335468:84 , srv570593514:84 , srv679238944:84 , srv694156289:84 , srv7040791:84 , srv72059144:84 , srv722009939:84 , srv748729315:84 , srv822256896:84 , srv869696087:84 , srv8915991:84 , srv906930057:84 , srv916622209:84 , srv924570053:84 , srv967723505:84 , srv1000267794:85 , srv1005788168:85 , srv1101290040:85 , srv110259878:85 , srv1102734357:85 , srv1203945641:85 , srv121638701:85 , srv1280766628:85 , srv1300974439:85 , srv1440223321:85 , srv1504018709:85 , srv1512403812:85 , srv1630696858:85 , srv1648566935:85 , srv1726814018:85 , srv1783322089:85 , srv1899502598:85 , srv2015432744:85 , srv2046811328:85 , srv2048950486:85 , srv2053899869:85 , srv2116210204:85 , srv219644809:85 , srv227744701:85 , srv238421735:85 , srv382373848:85 , srv427202821:85 , srv449286063:85 , srv498916979:85 , srv624985780:85 , srv62523307:85 , srv650932420:85 , srv682968112:85 , srv733503063:85 , srv763479044:85 , srv806733951:85 , srv819011036:85 , srv824656415:85 , srv847158180:85 , srv86796673:85 , srv876331554:85 , srv915516982:85 , srv996076198:85 , srv1428240504:86 , srv379281828:86 , srv553121678:86 , srv609887445:86 , srv536234229:640 }
2023-08-16 08:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv427202821=62, srv967723505=98, srv570593514=69, srv2072954566=44, srv806733951=84, srv536234229=66, srv1059128245=3, srv2031172082=40, srv667504155=74, srv819011036=85, srv1287572336=15, srv37174641=58, srv498916979=65, srv1432389981=20, srv86796673=89, srv7040791=78, srv411464922=61, srv1648566935=32, srv382373848=60, srv121638701=13, srv908270480=94, srv722009939=80, srv2092471043=45, srv924570053=97, srv379281828=59, srv1419616974=18, srv553121678=67, srv1169861291=8, srv822256896=86, srv1005788168=1, srv1314270655=17, srv1172761908=10, srv679238944=75, srv1845570786=37, srv996076198=99, srv1512403812=25, srv1471561231=23, srv1008176801=2, srv246416164=52, srv915516982=95, srv1504018709=24, srv1630696858=29, srv1632286796=30, srv733503063=81, srv1102734357=7, srv356672087=57, srv682968112=76, srv847158180=88, srv1594594124=27, srv650932420=73, srv1592752585=26, srv1827079851=36, srv1101290040=5, srv1300974439=16, srv624985780=71, srv1203945641=11, srv62523307=72, srv1783322089=35, srv763479044=83, srv1000267794=0, srv8915991=92, srv1280766628=14, srv1171409312=9, srv1204419426=12, srv310016121=54, srv694156289=77, srv609887445=70, srv1434580885=21, srv1726814018=33, srv876331554=91, srv2116210204=47, srv869696087=90, srv748729315=82, srv1899502598=38, srv556335468=68, srv110259878=6, srv906930057=93, srv353795964=56, srv473078126=64, srv340671675=55, srv28670266=53, srv214245181=48, srv219644809=49, srv227744701=50, srv1428240504=19, srv1440223321=22, srv1642756771=31, srv824656415=87, srv2015432744=39, srv1744573146=34, srv211053420=46, srv2046811328=41, srv72059144=79, srv2053899869=43, srv1100264458=4, srv238421735=51, srv916622209=96, srv1612481727=28, srv2048950486=42, srv449286063=63} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:16,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:16,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:16,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 08:53:16,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 08:53:16,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 08:53:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 08:53:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 08:53:16,164 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.776394154723469E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:16,165 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1432389981:82 , srv1434580885:83 , srv1612481727:83 , srv2092471043:83 , srv667504155:83 , srv908270480:83 , srv1008176801:84 , srv1059128245:84 , srv1100264458:84 , srv1169861291:84 , srv1171409312:84 , srv1172761908:84 , srv1204419426:84 , srv1287572336:84 , srv1314270655:84 , srv1419616974:84 , srv1471561231:84 , srv1592752585:84 , srv1594594124:84 , srv1632286796:84 , srv1642756771:84 , srv1744573146:84 , srv1827079851:84 , srv1845570786:84 , srv2031172082:84 , srv2072954566:84 , srv211053420:84 , srv214245181:84 , srv246416164:84 , srv28670266:84 , srv310016121:84 , srv340671675:84 , srv353795964:84 , srv356672087:84 , srv37174641:84 , srv411464922:84 , srv473078126:84 , srv556335468:84 , srv570593514:84 , srv679238944:84 , srv694156289:84 , srv7040791:84 , srv72059144:84 , srv722009939:84 , srv748729315:84 , srv822256896:84 , srv869696087:84 , srv8915991:84 , srv906930057:84 , srv916622209:84 , srv924570053:84 , srv967723505:84 , srv1000267794:85 , srv1005788168:85 , srv1101290040:85 , srv110259878:85 , srv1102734357:85 , srv1203945641:85 , srv121638701:85 , srv1280766628:85 , srv1300974439:85 , srv1440223321:85 , srv1504018709:85 , srv1512403812:85 , srv1630696858:85 , srv1648566935:85 , srv1726814018:85 , srv1783322089:85 , srv1899502598:85 , srv2015432744:85 , srv2046811328:85 , srv2048950486:85 , srv2053899869:85 , srv2116210204:85 , srv219644809:85 , srv227744701:85 , srv238421735:85 , srv382373848:85 , srv427202821:85 , srv449286063:85 , srv498916979:85 , srv624985780:85 , srv62523307:85 , srv650932420:85 , srv682968112:85 , srv733503063:85 , srv763479044:85 , srv806733951:85 , srv819011036:85 , srv824656415:85 , srv847158180:85 , srv86796673:85 , srv876331554:85 , srv915516982:85 , srv996076198:85 , srv1428240504:86 , srv379281828:86 , srv553121678:86 , srv609887445:86 , srv536234229:640 }
2023-08-16 08:53:16,202 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=314 (was 314), ProcessCount=174 (was 174), AvailableMemoryMB=1784 (was 2456)
2023-08-16 08:53:16,212 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=174, AvailableMemoryMB=1784
2023-08-16 08:53:16,213 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 08:53:16,213 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-16 08:53:16,224 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1043463151:84 , srv1074339774:84 , srv1106342376:84 , srv1185913218:84 , srv122609956:84 , srv1258888850:84 , srv1281340204:84 , srv132202239:84 , srv1375979459:84 , srv1593144963:84 , srv1594197478:84 , srv1769678141:84 , srv1832809246:84 , srv1870240223:84 , srv1874062954:84 , srv1876151538:84 , srv2003894711:84 , srv2092896538:84 , srv2113162258:84 , srv2129579956:84 , srv257167215:84 , srv417370790:84 , srv425512524:84 , srv444561027:84 , srv445932407:84 , srv495956201:84 , srv529459736:84 , srv620017357:84 , srv742505652:84 , srv1643107155:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-16 08:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:53:16,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv257167215=21, srv1594197478=10, srv2129579956=20, srv425512524=23, srv444561027=24, srv1870240223=14, srv2092896538=18, srv529459736=27, srv1106342376=2, srv1876151538=16, srv1074339774=1, srv620017357=28, srv1375979459=8, srv2113162258=19, srv1832809246=13, srv1258888850=5, srv417370790=22, srv1874062954=15, srv1643107155=11, srv742505652=29, srv1281340204=6, srv132202239=7, srv495956201=26, srv2003894711=17, srv1593144963=9, srv1185913218=3, srv445932407=25, srv1769678141=12, srv122609956=4, srv1043463151=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:53:16,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:53:16,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:53:16,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 08:53:16,254 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2139182306488374, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:53:16,254 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912867092623979, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2139182306488374, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-16 08:55:16,240 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5658888 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912867092623979 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:55:16,248 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1043463151:90 , srv1074339774:90 , srv1106342376:90 , srv1185913218:90 , srv122609956:90 , srv1258888850:90 , srv1281340204:90 , srv132202239:90 , srv1375979459:90 , srv1593144963:90 , srv1594197478:90 , srv1643107155:90 , srv1769678141:90 , srv1832809246:90 , srv1870240223:90 , srv1874062954:90 , srv1876151538:90 , srv2003894711:90 , srv2092896538:90 , srv2113162258:90 , srv2129579956:90 , srv257167215:90 , srv417370790:90 , srv425512524:90 , srv444561027:90 , srv445932407:90 , srv495956201:90 , srv529459736:90 , srv620017357:90 , srv742505652:90 }
2023-08-16 08:55:16,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 08:55:16,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv257167215=21, srv1594197478=10, srv2129579956=20, srv425512524=23, srv444561027=24, srv1870240223=14, srv2092896538=18, srv529459736=27, srv1106342376=2, srv1876151538=16, srv1074339774=1, srv620017357=28, srv1375979459=8, srv2113162258=19, srv1832809246=13, srv1258888850=5, srv417370790=22, srv1874062954=15, srv1643107155=11, srv742505652=29, srv1281340204=6, srv132202239=7, srv495956201=26, srv2003894711=17, srv1593144963=9, srv1185913218=3, srv445932407=25, srv1769678141=12, srv122609956=4, srv1043463151=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 08:55:16,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 08:55:16,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 08:55:16,267 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 08:55:16,267 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1043463151:90 , srv1074339774:90 , srv1106342376:90 , srv1185913218:90 , srv122609956:90 , srv1258888850:90 , srv1281340204:90 , srv132202239:90 , srv1375979459:90 , srv1593144963:90 , srv1594197478:90 , srv1643107155:90 , srv1769678141:90 , srv1832809246:90 , srv1870240223:90 , srv1874062954:90 , srv1876151538:90 , srv2003894711:90 , srv2092896538:90 , srv2113162258:90 , srv2129579956:90 , srv257167215:90 , srv417370790:90 , srv425512524:90 , srv444561027:90 , srv445932407:90 , srv495956201:90 , srv529459736:90 , srv620017357:90 , srv742505652:90 }
2023-08-16 08:55:16,282 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 314), ProcessCount=177 (was 174) - ProcessCount LEAK? -, AvailableMemoryMB=1763 (was 1784)