2023-04-11 11:00:21,388 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-11 11:00:21,644 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-11 11:00:21,733 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-11 11:00:21,780 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 11:00:21,785 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-11 11:00:21,795 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=239, ProcessCount=166, AvailableMemoryMB=8509
2023-04-11 11:00:21,797 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 11:00:21,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-11 11:00:21,900 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1028786399:84 , srv1068576447:84 , srv1080572092:84 , srv1081969685:84 , srv1108758038:84 , srv1113066570:84 , srv1121864156:84 , srv1134882043:84 , srv1148282752:84 , srv1174495086:84 , srv1181772116:84 , srv1198531514:84 , srv1210693959:84 , srv1224199972:84 , srv1232015716:84 , srv1267370326:84 , srv1288678513:84 , srv1337090134:84 , srv1341085557:84 , srv1346561142:84 , srv1366029478:84 , srv1384422073:84 , srv1411355426:84 , srv1412347806:84 , srv1414265035:84 , srv1431895960:84 , srv1432467055:84 , srv1437374460:84 , srv1458141140:84 , srv1564709198:84 , srv157317547:84 , srv1588377852:84 , srv1650707892:84 , srv1650981751:84 , srv1669276968:84 , srv1675673261:84 , srv1701332435:84 , srv1745986103:84 , srv1783568868:84 , srv1803555729:84 , srv1807028875:84 , srv1861889297:84 , srv1870554505:84 , srv1873859254:84 , srv1904096546:84 , srv1918181389:84 , srv1921721964:84 , srv1925523843:84 , srv1937741444:84 , srv1946189571:84 , srv1956995739:84 , srv1968504744:84 , srv2012768865:84 , srv2026971377:84 , srv2108383898:84 , srv2127358392:84 , srv308920944:84 , srv315068065:84 , srv323871268:84 , srv339488776:84 , srv373336381:84 , srv377802823:84 , srv378775728:84 , srv388407414:84 , srv438583220:84 , srv444383969:84 , srv449559237:84 , srv455133758:84 , srv462852917:84 , srv47140062:84 , srv51682918:84 , srv526588469:84 , srv591964427:84 , srv593735770:84 , srv603512296:84 , srv612240111:84 , srv61894179:84 , srv62556999:84 , srv642885199:84 , srv658870661:84 , srv66333146:84 , srv670135593:84 , srv718057268:84 , srv719510175:84 , srv724731359:84 , srv738510332:84 , srv753538025:84 , srv754661075:84 , srv813666163:84 , srv821085022:84 , srv824441219:84 , srv825478478:84 , srv882044917:84 , srv898094798:84 , srv950016863:84 , srv95986839:84 , srv970168808:84 , srv976574678:84 , srv981699428:84 , srv2108411027:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-11 11:00:21,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:21,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:21,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:21,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:21,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:21,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:21,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:21,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:21,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:21,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:22,156 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:22,156 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 11:00:22,156 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 11:00:22,590 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 575 ms to try 5000 different iterations.  Found a solution that moves 4652 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18495233959119053. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05648515194785609, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5168888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18008333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:22,611 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1366029478:82 , srv1414265035:82 , srv1669276968:82 , srv2026971377:82 , srv455133758:82 , srv612240111:82 , srv95986839:82 , srv970168808:82 , srv1113066570:83 , srv1341085557:83 , srv157317547:83 , srv2012768865:83 , srv388407414:83 , srv591964427:83 , srv1134882043:84 , srv1232015716:84 , srv1650981751:84 , srv1803555729:84 , srv1921721964:84 , srv1956995739:84 , srv315068065:84 , srv670135593:84 , srv1028786399:85 , srv1068576447:85 , srv1121864156:85 , srv1181772116:85 , srv1198531514:85 , srv1224199972:85 , srv1807028875:85 , srv1861889297:85 , srv1918181389:85 , srv2108383898:85 , srv308920944:85 , srv438583220:85 , srv449559237:85 , srv658870661:85 , srv66333146:85 , srv825478478:85 , srv898094798:85 , srv1081969685:86 , srv1148282752:86 , srv1337090134:86 , srv1412347806:86 , srv1745986103:86 , srv1870554505:86 , srv1873859254:86 , srv1904096546:86 , srv1925523843:86 , srv1937741444:86 , srv2127358392:86 , srv323871268:86 , srv51682918:86 , srv62556999:86 , srv719510175:86 , srv813666163:86 , srv882044917:86 , srv1080572092:87 , srv1174495086:87 , srv1288678513:87 , srv1432467055:87 , srv1458141140:87 , srv1588377852:87 , srv1783568868:87 , srv1946189571:87 , srv373336381:87 , srv377802823:87 , srv378775728:87 , srv47140062:87 , srv61894179:87 , srv642885199:87 , srv718057268:87 , srv753538025:87 , srv754661075:87 , srv1701332435:88 , srv444383969:88 , srv593735770:88 , srv603512296:88 , srv738510332:88 , srv950016863:88 , srv976574678:88 , srv1267370326:89 , srv1411355426:89 , srv1437374460:89 , srv1564709198:89 , srv1968504744:89 , srv462852917:89 , srv724731359:89 , srv821085022:89 , srv824441219:89 , srv1108758038:90 , srv1346561142:90 , srv1431895960:90 , srv339488776:90 , srv1650707892:91 , srv526588469:91 , srv1384422073:92 , srv1675673261:92 , srv981699428:93 , srv1210693959:95 , srv2108411027:457 }
2023-04-11 11:00:22,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:22,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:22,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:22,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:22,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:22,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:22,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:22,725 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05648515194785609, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18008333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:22,726 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 11:00:22,726 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18491976559097245, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04129014312993761, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05648515194785609, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04129014312993761, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18008333333333335, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 11:00:22,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 258 ms to try 5000 different iterations.  Found a solution that moves 2664 regions; Going from a computed imbalance of 0.18491976559097245 to a new imbalance of 0.014942560303076545. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021704979220399517); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03384788047050626, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021704979220399517); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:22,914 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1675673261:79 , srv981699428:79 , srv1210693959:80 , srv1650707892:80 , srv1384422073:81 , srv1564709198:81 , srv1411355426:82 , srv1267370326:83 , srv1437374460:83 , srv1946189571:83 , srv1968504744:83 , srv339488776:83 , srv377802823:83 , srv462852917:83 , srv526588469:83 , srv593735770:83 , srv603512296:83 , srv642885199:83 , srv724731359:83 , srv738510332:83 , srv753538025:83 , srv821085022:83 , srv1108758038:84 , srv1431895960:84 , srv444383969:84 , srv61894179:84 , srv719510175:84 , srv754661075:84 , srv824441219:84 , srv1080572092:85 , srv1346561142:85 , srv1432467055:85 , srv1701332435:85 , srv1745986103:85 , srv1783568868:85 , srv1861889297:85 , srv1873859254:85 , srv378775728:85 , srv718057268:85 , srv813666163:85 , srv950016863:85 , srv976574678:85 , srv1081969685:86 , srv1224199972:86 , srv1288678513:86 , srv1458141140:86 , srv1588377852:86 , srv1807028875:86 , srv1937741444:86 , srv1956995739:86 , srv2127358392:86 , srv308920944:86 , srv323871268:86 , srv373336381:86 , srv449559237:86 , srv47140062:86 , srv51682918:86 , srv658870661:86 , srv882044917:86 , srv898094798:86 , srv1028786399:87 , srv1148282752:87 , srv1174495086:87 , srv1181772116:87 , srv1198531514:87 , srv1232015716:87 , srv1337090134:87 , srv1412347806:87 , srv1650981751:87 , srv1870554505:87 , srv2108383898:87 , srv315068065:87 , srv438583220:87 , srv1068576447:88 , srv1113066570:88 , srv1121864156:88 , srv1134882043:88 , srv1904096546:88 , srv1918181389:88 , srv1925523843:88 , srv2012768865:88 , srv388407414:88 , srv591964427:88 , srv66333146:88 , srv970168808:88 , srv1341085557:89 , srv1366029478:89 , srv1414265035:89 , srv157317547:89 , srv1803555729:89 , srv62556999:89 , srv95986839:89 , srv1669276968:90 , srv1921721964:90 , srv455133758:90 , srv670135593:90 , srv825478478:90 , srv2026971377:91 , srv612240111:91 , srv2108411027:510 }
2023-04-11 11:00:22,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:22,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:22,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:22,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:22,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:22,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:22,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:22,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:22,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:22,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:22,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:22,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:22,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:22,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:23,022 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021704979220399517); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03384788047050626, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021704979220399517); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,023 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 11:00:23,023 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01492390657638245, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021704979220399517); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03384788047050626, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021704979220399517); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044583333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 11:00:23,173 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 224 ms to try 5000 different iterations.  Found a solution that moves 995 regions; Going from a computed imbalance of 0.01492390657638245 to a new imbalance of 0.0014435384880694205. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006777970963742381); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.11055555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006777970963742381); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0010833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,174 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1210693959:82 , srv1921721964:82 , srv1121864156:83 , srv1346561142:83 , srv1925523843:83 , srv62556999:83 , srv825478478:83 , srv1108758038:84 , srv1174495086:84 , srv1384422073:84 , srv1431895960:84 , srv1588377852:84 , srv1701332435:84 , srv1783568868:84 , srv1803555729:84 , srv1870554505:84 , srv1904096546:84 , srv1918181389:84 , srv47140062:84 , srv66333146:84 , srv670135593:84 , srv882044917:84 , srv976574678:84 , srv1028786399:85 , srv1068576447:85 , srv1134882043:85 , srv1148282752:85 , srv1181772116:85 , srv1267370326:85 , srv1288678513:85 , srv1341085557:85 , srv1411355426:85 , srv1412347806:85 , srv1432467055:85 , srv1458141140:85 , srv157317547:85 , srv1669276968:85 , srv1873859254:85 , srv1937741444:85 , srv1968504744:85 , srv2026971377:85 , srv2108383898:85 , srv2127358392:85 , srv339488776:85 , srv373336381:85 , srv444383969:85 , srv526588469:85 , srv612240111:85 , srv718057268:85 , srv724731359:85 , srv821085022:85 , srv824441219:85 , srv950016863:85 , srv1080572092:86 , srv1081969685:86 , srv1198531514:86 , srv1224199972:86 , srv1232015716:86 , srv1414265035:86 , srv1437374460:86 , srv1650707892:86 , srv1650981751:86 , srv1675673261:86 , srv1807028875:86 , srv1956995739:86 , srv308920944:86 , srv315068065:86 , srv323871268:86 , srv378775728:86 , srv438583220:86 , srv449559237:86 , srv455133758:86 , srv462852917:86 , srv51682918:86 , srv591964427:86 , srv603512296:86 , srv61894179:86 , srv658870661:86 , srv738510332:86 , srv754661075:86 , srv813666163:86 , srv898094798:86 , srv981699428:86 , srv1113066570:87 , srv1337090134:87 , srv1366029478:87 , srv1564709198:87 , srv1745986103:87 , srv1946189571:87 , srv2012768865:87 , srv377802823:87 , srv593735770:87 , srv719510175:87 , srv753538025:87 , srv95986839:87 , srv1861889297:88 , srv388407414:88 , srv642885199:88 , srv970168808:88 , srv2108411027:551 }
2023-04-11 11:00:23,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:23,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:23,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:23,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:23,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:23,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:23,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:23,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:23,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:23,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:23,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:23,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:23,269 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006777970963742381); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006777970963742381); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0010833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,270 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 11:00:23,270 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0014365713491577747, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006777970963742381); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006777970963742381); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0010833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 11:00:23,475 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 274 ms to try 5000 different iterations.  Found a solution that moves 254 regions; Going from a computed imbalance of 0.0014365713491577747 to a new imbalance of 4.407277800785344E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.02822222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,476 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1337090134:82 , srv378775728:83 , srv388407414:83 , srv438583220:83 , srv526588469:83 , srv1068576447:84 , srv1081969685:84 , srv1108758038:84 , srv1113066570:84 , srv1148282752:84 , srv1174495086:84 , srv1198531514:84 , srv1267370326:84 , srv1288678513:84 , srv1346561142:84 , srv1366029478:84 , srv1412347806:84 , srv1431895960:84 , srv1437374460:84 , srv1458141140:84 , srv1701332435:84 , srv1745986103:84 , srv1803555729:84 , srv1861889297:84 , srv1904096546:84 , srv1946189571:84 , srv1968504744:84 , srv2026971377:84 , srv323871268:84 , srv339488776:84 , srv373336381:84 , srv455133758:84 , srv462852917:84 , srv51682918:84 , srv591964427:84 , srv593735770:84 , srv612240111:84 , srv62556999:84 , srv642885199:84 , srv670135593:84 , srv824441219:84 , srv825478478:84 , srv950016863:84 , srv970168808:84 , srv981699428:84 , srv1028786399:85 , srv1080572092:85 , srv1134882043:85 , srv1181772116:85 , srv1224199972:85 , srv1232015716:85 , srv1341085557:85 , srv1384422073:85 , srv1414265035:85 , srv1432467055:85 , srv1564709198:85 , srv157317547:85 , srv1588377852:85 , srv1650707892:85 , srv1669276968:85 , srv1675673261:85 , srv1807028875:85 , srv1870554505:85 , srv1873859254:85 , srv1925523843:85 , srv2012768865:85 , srv2108383898:85 , srv308920944:85 , srv315068065:85 , srv377802823:85 , srv444383969:85 , srv449559237:85 , srv47140062:85 , srv603512296:85 , srv61894179:85 , srv658870661:85 , srv66333146:85 , srv718057268:85 , srv719510175:85 , srv724731359:85 , srv738510332:85 , srv753538025:85 , srv754661075:85 , srv813666163:85 , srv821085022:85 , srv898094798:85 , srv95986839:85 , srv976574678:85 , srv1121864156:86 , srv1210693959:86 , srv1650981751:86 , srv1783568868:86 , srv1918181389:86 , srv1921721964:86 , srv1937741444:86 , srv1956995739:86 , srv2127358392:86 , srv882044917:86 , srv1411355426:87 , srv2108411027:624 }
2023-04-11 11:00:23,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:23,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:23,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:23,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:23,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:23,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:23,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:23,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:23,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:23,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:23,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:23,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:23,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:23,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:23,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:23,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,573 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-11 11:00:23,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.229423199422726E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-11 11:00:23,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 202 ms to try 5000 different iterations.  Found a solution that moves 35 regions; Going from a computed imbalance of 4.229423199422726E-5 to a new imbalance of 7.318314534427544E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0038888888888888888); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,705 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1080572092:83 , srv1210693959:83 , srv1411355426:83 , srv1650981751:83 , srv1918181389:83 , srv1937741444:83 , srv2012768865:83 , srv2127358392:83 , srv95986839:83 , srv1028786399:84 , srv1068576447:84 , srv1081969685:84 , srv1108758038:84 , srv1113066570:84 , srv1121864156:84 , srv1134882043:84 , srv1148282752:84 , srv1174495086:84 , srv1181772116:84 , srv1198531514:84 , srv1224199972:84 , srv1232015716:84 , srv1288678513:84 , srv1337090134:84 , srv1341085557:84 , srv1346561142:84 , srv1366029478:84 , srv1384422073:84 , srv1412347806:84 , srv1414265035:84 , srv1431895960:84 , srv1432467055:84 , srv1437374460:84 , srv1458141140:84 , srv1564709198:84 , srv157317547:84 , srv1588377852:84 , srv1650707892:84 , srv1669276968:84 , srv1675673261:84 , srv1745986103:84 , srv1783568868:84 , srv1807028875:84 , srv1861889297:84 , srv1870554505:84 , srv1904096546:84 , srv1921721964:84 , srv1925523843:84 , srv1956995739:84 , srv2026971377:84 , srv2108383898:84 , srv308920944:84 , srv315068065:84 , srv323871268:84 , srv339488776:84 , srv373336381:84 , srv377802823:84 , srv388407414:84 , srv444383969:84 , srv449559237:84 , srv455133758:84 , srv462852917:84 , srv47140062:84 , srv51682918:84 , srv526588469:84 , srv593735770:84 , srv603512296:84 , srv612240111:84 , srv61894179:84 , srv62556999:84 , srv642885199:84 , srv658870661:84 , srv66333146:84 , srv670135593:84 , srv718057268:84 , srv719510175:84 , srv724731359:84 , srv738510332:84 , srv753538025:84 , srv754661075:84 , srv813666163:84 , srv821085022:84 , srv824441219:84 , srv825478478:84 , srv882044917:84 , srv898094798:84 , srv950016863:84 , srv970168808:84 , srv976574678:84 , srv981699428:84 , srv1267370326:85 , srv1701332435:85 , srv1803555729:85 , srv1873859254:85 , srv1946189571:85 , srv1968504744:85 , srv378775728:85 , srv438583220:85 , srv591964427:85 , srv2108411027:684 }
2023-04-11 11:00:23,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:23,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1121864156=6, srv1431895960=25, srv970168808=97, srv1873859254=43, srv323871268=59, srv315068065=58, srv1458141140=28, srv753538025=87, srv1412347806=23, srv2026971377=53, srv1414265035=24, srv444383969=66, srv1783568868=38, srv1870554505=42, srv1946189571=49, srv2108383898=54, srv1745986103=37, srv61894179=77, srv1068576447=1, srv1968504744=51, srv719510175=84, srv981699428=99, srv1564709198=29, srv1224199972=13, srv47140062=70, srv378775728=63, srv95986839=96, srv388407414=64, srv449559237=67, srv1108758038=4, srv1861889297=41, srv1232015716=14, srv462852917=69, srv898094798=94, srv1411355426=22, srv1210693959=12, srv1701332435=36, srv2012768865=52, srv1675673261=35, srv1198531514=11, srv976574678=98, srv1669276968=34, srv1918181389=45, srv373336381=61, srv1650981751=33, srv882044917=93, srv1080572092=2, srv1650707892=32, srv824441219=91, srv1437374460=27, srv66333146=81, srv339488776=60, srv526588469=72, srv1341085557=18, srv813666163=89, srv62556999=78, srv1588377852=31, srv1904096546=44, srv738510332=86, srv1113066570=5, srv591964427=73, srv1267370326=15, srv1366029478=20, srv1956995739=50, srv724731359=85, srv754661075=88, srv1925523843=47, srv377802823=62, srv642885199=79, srv1028786399=0, srv1384422073=21, srv1181772116=10, srv950016863=95, srv1288678513=16, srv1432467055=26, srv1937741444=48, srv1134882043=7, srv308920944=57, srv455133758=68, srv658870661=80, srv1803555729=39, srv821085022=90, srv1807028875=40, srv2127358392=56, srv1148282752=8, srv593735770=74, srv603512296=75, srv612240111=76, srv670135593=82, srv1337090134=17, srv2108411027=55, srv1174495086=9, srv1921721964=46, srv1081969685=3, srv1346561142=19, srv157317547=30, srv825478478=92, srv438583220=65, srv51682918=71, srv718057268=83} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:23,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:23,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-11 11:00:23,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-11 11:00:23,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-11 11:00:23,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-11 11:00:23,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-11 11:00:23,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-11 11:00:23,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-11 11:00:23,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-11 11:00:23,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-11 11:00:23,805 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,805 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1080572092:83 , srv1210693959:83 , srv1411355426:83 , srv1650981751:83 , srv1918181389:83 , srv1937741444:83 , srv2012768865:83 , srv2127358392:83 , srv95986839:83 , srv1028786399:84 , srv1068576447:84 , srv1081969685:84 , srv1108758038:84 , srv1113066570:84 , srv1121864156:84 , srv1134882043:84 , srv1148282752:84 , srv1174495086:84 , srv1181772116:84 , srv1198531514:84 , srv1224199972:84 , srv1232015716:84 , srv1288678513:84 , srv1337090134:84 , srv1341085557:84 , srv1346561142:84 , srv1366029478:84 , srv1384422073:84 , srv1412347806:84 , srv1414265035:84 , srv1431895960:84 , srv1432467055:84 , srv1437374460:84 , srv1458141140:84 , srv1564709198:84 , srv157317547:84 , srv1588377852:84 , srv1650707892:84 , srv1669276968:84 , srv1675673261:84 , srv1745986103:84 , srv1783568868:84 , srv1807028875:84 , srv1861889297:84 , srv1870554505:84 , srv1904096546:84 , srv1921721964:84 , srv1925523843:84 , srv1956995739:84 , srv2026971377:84 , srv2108383898:84 , srv308920944:84 , srv315068065:84 , srv323871268:84 , srv339488776:84 , srv373336381:84 , srv377802823:84 , srv388407414:84 , srv444383969:84 , srv449559237:84 , srv455133758:84 , srv462852917:84 , srv47140062:84 , srv51682918:84 , srv526588469:84 , srv593735770:84 , srv603512296:84 , srv612240111:84 , srv61894179:84 , srv62556999:84 , srv642885199:84 , srv658870661:84 , srv66333146:84 , srv670135593:84 , srv718057268:84 , srv719510175:84 , srv724731359:84 , srv738510332:84 , srv753538025:84 , srv754661075:84 , srv813666163:84 , srv821085022:84 , srv824441219:84 , srv825478478:84 , srv882044917:84 , srv898094798:84 , srv950016863:84 , srv970168808:84 , srv976574678:84 , srv981699428:84 , srv1267370326:85 , srv1701332435:85 , srv1803555729:85 , srv1873859254:85 , srv1946189571:85 , srv1968504744:85 , srv378775728:85 , srv438583220:85 , srv591964427:85 , srv2108411027:684 }
2023-04-11 11:00:23,826 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=239 (was 239), ProcessCount=166 (was 166), AvailableMemoryMB=8021 (was 8509)
2023-04-11 11:00:23,832 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=239, ProcessCount=166, AvailableMemoryMB=8021
2023-04-11 11:00:23,833 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-11 11:00:23,833 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-11 11:00:23,841 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035761578:84 , srv1055353872:84 , srv1087902392:84 , srv113352466:84 , srv1176260079:84 , srv1203548471:84 , srv1239352313:84 , srv1240485167:84 , srv1345013487:84 , srv1503408909:84 , srv1528334512:84 , srv1562748285:84 , srv1723309472:84 , srv1822659299:84 , srv1857751855:84 , srv1861555184:84 , srv255230759:84 , srv276998962:84 , srv35399709:84 , srv400193061:84 , srv417009853:84 , srv443349379:84 , srv444541681:84 , srv457164759:84 , srv662540603:84 , srv665515919:84 , srv773950739:84 , srv968775:84 , srv981812243:84 , srv461378362:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-11 11:00:23,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:00:23,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1055353872=1, srv1176260079=4, srv1528334512=10, srv443349379=21, srv1503408909=9, srv662540603=25, srv968775=28, srv1562748285=11, srv1822659299=13, srv444541681=22, srv665515919=26, srv1240485167=7, srv1345013487=8, srv276998962=17, srv1203548471=5, srv417009853=20, srv35399709=18, srv457164759=23, srv1035761578=0, srv461378362=24, srv1723309472=12, srv773950739=27, srv1239352313=6, srv1087902392=2, srv113352466=3, srv255230759=16, srv981812243=29, srv400193061=19, srv1857751855=14, srv1861555184=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:00:23,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:00:23,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:00:23,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:00:23,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:00:23,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:00:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:00:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:00:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:00:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:00:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:00:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-11 11:00:23,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0941840423634837, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:00:23,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912489814106376, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0941840423634837, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-11 11:02:23,858 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10026996 different iterations.  Found a solution that moves 1844 regions; Going from a computed imbalance of 0.9912489814106376 to a new imbalance of 4.303987990979898E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.682962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:02:23,863 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035761578:90 , srv1055353872:90 , srv1087902392:90 , srv113352466:90 , srv1176260079:90 , srv1203548471:90 , srv1239352313:90 , srv1240485167:90 , srv1345013487:90 , srv1503408909:90 , srv1528334512:90 , srv1562748285:90 , srv1723309472:90 , srv1822659299:90 , srv1857751855:90 , srv1861555184:90 , srv255230759:90 , srv276998962:90 , srv35399709:90 , srv400193061:90 , srv417009853:90 , srv443349379:90 , srv444541681:90 , srv457164759:90 , srv461378362:90 , srv662540603:90 , srv665515919:90 , srv773950739:90 , srv968775:90 , srv981812243:90 }
2023-04-11 11:02:23,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-11 11:02:23,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1055353872=1, srv1176260079=4, srv1528334512=10, srv443349379=21, srv1503408909=9, srv662540603=25, srv968775=28, srv1562748285=11, srv1822659299=13, srv444541681=22, srv665515919=26, srv1240485167=7, srv1345013487=8, srv276998962=17, srv1203548471=5, srv417009853=20, srv35399709=18, srv457164759=23, srv1035761578=0, srv461378362=24, srv1723309472=12, srv773950739=27, srv1239352313=6, srv1087902392=2, srv113352466=3, srv255230759=16, srv981812243=29, srv400193061=19, srv1857751855=14, srv1861555184=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-11 11:02:23,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-11 11:02:23,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-11 11:02:23,877 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-11 11:02:23,877 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035761578:90 , srv1055353872:90 , srv1087902392:90 , srv113352466:90 , srv1176260079:90 , srv1203548471:90 , srv1239352313:90 , srv1240485167:90 , srv1345013487:90 , srv1503408909:90 , srv1528334512:90 , srv1562748285:90 , srv1723309472:90 , srv1822659299:90 , srv1857751855:90 , srv1861555184:90 , srv255230759:90 , srv276998962:90 , srv35399709:90 , srv400193061:90 , srv417009853:90 , srv443349379:90 , srv444541681:90 , srv457164759:90 , srv461378362:90 , srv662540603:90 , srv665515919:90 , srv773950739:90 , srv968775:90 , srv981812243:90 }
2023-04-11 11:02:23,887 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=163 (was 239), ProcessCount=164 (was 166), AvailableMemoryMB=8022 (was 8021) - AvailableMemoryMB LEAK? -