2023-07-12 13:48:48,166 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-12 13:48:48,423 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-12 13:48:48,526 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-12 13:48:48,590 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 13:48:48,597 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-12 13:48:48,609 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=434, ProcessCount=174, AvailableMemoryMB=3785
2023-07-12 13:48:48,611 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 13:48:48,612 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-12 13:48:48,769 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100837275:84 , srv1035731114:84 , srv1035756050:84 , srv1101677117:84 , srv1175575492:84 , srv1191235578:84 , srv1193237605:84 , srv1198279832:84 , srv1223086913:84 , srv1289097621:84 , srv1290815854:84 , srv131952737:84 , srv136426960:84 , srv1392568048:84 , srv1427997487:84 , srv1456614271:84 , srv1472380787:84 , srv1521855963:84 , srv1528553455:84 , srv1531247696:84 , srv1556118384:84 , srv1564836788:84 , srv1589406299:84 , srv1603758642:84 , srv1620890955:84 , srv1667534369:84 , srv169488281:84 , srv1699962470:84 , srv1701534496:84 , srv170713757:84 , srv1709017904:84 , srv1723705619:84 , srv1745030600:84 , srv1771287619:84 , srv177958343:84 , srv1781807016:84 , srv1789020538:84 , srv1795959064:84 , srv1829135175:84 , srv1841562062:84 , srv1876694353:84 , srv1902113792:84 , srv1933660610:84 , srv1939351619:84 , srv1961713763:84 , srv1962093843:84 , srv199349859:84 , srv2033003391:84 , srv2056536767:84 , srv2095691565:84 , srv2097661951:84 , srv2106922214:84 , srv2113325377:84 , srv2123514174:84 , srv2129205567:84 , srv2134279539:84 , srv2146295160:84 , srv222609149:84 , srv229783671:84 , srv236774952:84 , srv247536349:84 , srv249948106:84 , srv257953247:84 , srv283429645:84 , srv30542737:84 , srv308326321:84 , srv340578733:84 , srv358730471:84 , srv372496195:84 , srv393558866:84 , srv398662393:84 , srv41382483:84 , srv456519577:84 , srv480392790:84 , srv480579889:84 , srv484958044:84 , srv493715296:84 , srv509255728:84 , srv624152668:84 , srv642241938:84 , srv650040947:84 , srv661673521:84 , srv706399119:84 , srv715342987:84 , srv732699301:84 , srv75680972:84 , srv768338738:84 , srv812080159:84 , srv845466672:84 , srv865899833:84 , srv880169726:84 , srv904513412:84 , srv919560268:84 , srv924035170:84 , srv951304711:84 , srv957321778:84 , srv960204052:84 , srv978897302:84 , srv980495516:84 , srv1375147799:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-12 13:48:48,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:48,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962093843=46, srv222609149=58, srv398662393=71, srv1521855963=18, srv642241938=80, srv957321778=96, srv493715296=77, srv2095691565=50, srv706399119=83, srv1841562062=40, srv1193237605=6, srv484958044=76, srv1701534496=29, srv2056536767=49, srv283429645=64, srv229783671=59, srv249948106=62, srv1876694353=41, srv308326321=66, srv1829135175=39, srv1603758642=24, srv1699962470=28, srv880169726=91, srv1961713763=45, srv919560268=93, srv1456614271=16, srv2146295160=57, srv1902113792=42, srv2129205567=55, srv661673521=82, srv2106922214=52, srv1035756050=2, srv340578733=67, srv1101677117=3, srv236774952=60, srv257953247=63, srv456519577=73, srv624152668=79, srv1191235578=5, srv2123514174=54, srv951304711=95, srv845466672=89, srv1667534369=26, srv1771287619=34, srv1723705619=32, srv650040947=81, srv715342987=84, srv1290815854=10, srv2113325377=53, srv1939351619=44, srv1531247696=20, srv177958343=35, srv1175575492=4, srv1589406299=23, srv480579889=75, srv812080159=88, srv30542737=65, srv732699301=85, srv1745030600=33, srv393558866=70, srv480392790=74, srv1781807016=36, srv865899833=90, srv1795959064=38, srv924035170=94, srv247536349=61, srv768338738=87, srv2033003391=48, srv372496195=69, srv1789020538=37, srv1556118384=21, srv41382483=72, srv131952737=11, srv1223086913=8, srv1933660610=43, srv2134279539=56, srv1472380787=17, srv1564836788=22, srv1198279832=7, srv980495516=99, srv960204052=97, srv2097661951=51, srv1620890955=25, srv1427997487=15, srv169488281=27, srv1709017904=31, srv170713757=30, srv1035731114=1, srv136426960=12, srv199349859=47, srv75680972=86, srv1528553455=19, srv358730471=68, srv978897302=98, srv509255728=78, srv1375147799=13, srv904513412=92, srv1289097621=9, srv100837275=0, srv1392568048=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:48,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:48,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:48,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:48,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:48,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:48,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 13:48:48,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 13:48:48,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 13:48:48,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 13:48:48,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 13:48:48,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 13:48:48,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 13:48:48,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 13:48:48,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 13:48:48,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 13:48:48,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 13:48:48,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 13:48:48,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 13:48:48,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 13:48:48,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 13:48:48,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 13:48:48,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 13:48:48,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 13:48:48,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 13:48:48,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 13:48:48,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 13:48:48,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 13:48:49,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:49,155 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 13:48:49,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 13:48:49,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 478 ms to try 5000 different iterations.  Found a solution that moves 4691 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18415608410955644. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377110389562459, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336278648009597, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5212222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0377110389562459, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:49,476 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1456614271:80 , srv100837275:82 , srv222609149:82 , srv924035170:82 , srv980495516:82 , srv1193237605:83 , srv1223086913:83 , srv131952737:83 , srv1528553455:83 , srv1603758642:83 , srv283429645:83 , srv1564836788:84 , srv177958343:84 , srv2106922214:84 , srv229783671:84 , srv236774952:84 , srv41382483:84 , srv493715296:84 , srv951304711:84 , srv1175575492:85 , srv1198279832:85 , srv1427997487:85 , srv1472380787:85 , srv1709017904:85 , srv2123514174:85 , srv247536349:85 , srv509255728:85 , srv624152668:85 , srv75680972:85 , srv812080159:85 , srv865899833:85 , srv1035756050:86 , srv1290815854:86 , srv1392568048:86 , srv1556118384:86 , srv1620890955:86 , srv169488281:86 , srv1701534496:86 , srv1723705619:86 , srv1745030600:86 , srv1771287619:86 , srv1795959064:86 , srv1876694353:86 , srv1933660610:86 , srv2033003391:86 , srv2095691565:86 , srv257953247:86 , srv340578733:86 , srv393558866:86 , srv480579889:86 , srv484958044:86 , srv650040947:86 , srv661673521:86 , srv1531247696:87 , srv1667534369:87 , srv170713757:87 , srv1781807016:87 , srv1829135175:87 , srv1841562062:87 , srv1962093843:87 , srv199349859:87 , srv30542737:87 , srv358730471:87 , srv642241938:87 , srv706399119:87 , srv957321778:87 , srv1101677117:88 , srv136426960:88 , srv1789020538:88 , srv1939351619:88 , srv2129205567:88 , srv2146295160:88 , srv249948106:88 , srv398662393:88 , srv732699301:88 , srv768338738:88 , srv880169726:88 , srv978897302:88 , srv1035731114:89 , srv1289097621:89 , srv1521855963:89 , srv1589406299:89 , srv1699962470:89 , srv456519577:89 , srv480392790:89 , srv715342987:89 , srv960204052:89 , srv1191235578:90 , srv2134279539:90 , srv308326321:90 , srv372496195:90 , srv845466672:90 , srv904513412:90 , srv2097661951:92 , srv2113325377:92 , srv1902113792:93 , srv1961713763:93 , srv2056536767:93 , srv919560268:94 , srv1375147799:425 }
2023-07-12 13:48:49,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:49,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962093843=46, srv222609149=58, srv398662393=71, srv1521855963=18, srv642241938=80, srv957321778=96, srv493715296=77, srv2095691565=50, srv706399119=83, srv1841562062=40, srv1193237605=6, srv484958044=76, srv1701534496=29, srv2056536767=49, srv283429645=64, srv229783671=59, srv249948106=62, srv1876694353=41, srv308326321=66, srv1829135175=39, srv1603758642=24, srv1699962470=28, srv880169726=91, srv1961713763=45, srv919560268=93, srv1456614271=16, srv2146295160=57, srv1902113792=42, srv2129205567=55, srv661673521=82, srv2106922214=52, srv1035756050=2, srv340578733=67, srv1101677117=3, srv236774952=60, srv257953247=63, srv456519577=73, srv624152668=79, srv1191235578=5, srv2123514174=54, srv951304711=95, srv845466672=89, srv1667534369=26, srv1771287619=34, srv1723705619=32, srv650040947=81, srv715342987=84, srv1290815854=10, srv2113325377=53, srv1939351619=44, srv1531247696=20, srv177958343=35, srv1175575492=4, srv1589406299=23, srv480579889=75, srv812080159=88, srv30542737=65, srv732699301=85, srv1745030600=33, srv393558866=70, srv480392790=74, srv1781807016=36, srv865899833=90, srv1795959064=38, srv924035170=94, srv247536349=61, srv768338738=87, srv2033003391=48, srv372496195=69, srv1789020538=37, srv1556118384=21, srv41382483=72, srv131952737=11, srv1223086913=8, srv1933660610=43, srv2134279539=56, srv1472380787=17, srv1564836788=22, srv1198279832=7, srv980495516=99, srv960204052=97, srv2097661951=51, srv1620890955=25, srv1427997487=15, srv169488281=27, srv1709017904=31, srv170713757=30, srv1035731114=1, srv136426960=12, srv199349859=47, srv75680972=86, srv1528553455=19, srv358730471=68, srv978897302=98, srv509255728=78, srv1375147799=13, srv904513412=92, srv1289097621=9, srv100837275=0, srv1392568048=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:49,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:49,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:49,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:49,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 13:48:49,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 13:48:49,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 13:48:49,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 13:48:49,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 13:48:49,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 13:48:49,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 13:48:49,615 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377110389562459, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336278648009597, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0377110389562459, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:49,615 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 13:48:49,616 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18412323702550162, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377110389562459, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05336278648009597, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0377110389562459, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 13:48:49,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 315 ms to try 5000 different iterations.  Found a solution that moves 2649 regions; Going from a computed imbalance of 0.18412323702550162 to a new imbalance of 0.014670233743620885. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018817783906348487); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02979541163389459, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29433333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018817783906348487); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:49,847 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv919560268:76 , srv1902113792:80 , srv2097661951:80 , srv2113325377:80 , srv308326321:80 , srv1961713763:81 , srv456519577:81 , srv1789020538:82 , srv2134279539:82 , srv372496195:82 , srv845466672:82 , srv904513412:82 , srv1191235578:83 , srv1521855963:83 , srv1829135175:83 , srv1939351619:83 , srv2056536767:83 , srv249948106:83 , srv480392790:83 , srv715342987:83 , srv1175575492:84 , srv1589406299:84 , srv1701534496:84 , srv1781807016:84 , srv2095691565:84 , srv2146295160:84 , srv257953247:84 , srv642241938:84 , srv732699301:84 , srv960204052:84 , srv978897302:84 , srv1101677117:85 , srv1198279832:85 , srv1289097621:85 , srv1620890955:85 , srv1699962470:85 , srv1723705619:85 , srv1745030600:85 , srv1771287619:85 , srv199349859:85 , srv2129205567:85 , srv340578733:85 , srv624152668:85 , srv650040947:85 , srv706399119:85 , srv768338738:85 , srv1035756050:86 , srv1290815854:86 , srv1392568048:86 , srv1667534369:86 , srv169488281:86 , srv1795959064:86 , srv1841562062:86 , srv1933660610:86 , srv2123514174:86 , srv236774952:86 , srv358730471:86 , srv484958044:86 , srv880169726:86 , srv957321778:86 , srv1035731114:87 , srv136426960:87 , srv1427997487:87 , srv1472380787:87 , srv1556118384:87 , srv170713757:87 , srv1709017904:87 , srv1876694353:87 , srv1962093843:87 , srv2033003391:87 , srv2106922214:87 , srv30542737:87 , srv393558866:87 , srv398662393:87 , srv509255728:87 , srv865899833:87 , srv131952737:88 , srv1564836788:88 , srv229783671:88 , srv247536349:88 , srv493715296:88 , srv951304711:88 , srv1193237605:89 , srv1528553455:89 , srv1531247696:89 , srv177958343:89 , srv222609149:89 , srv41382483:89 , srv480579889:89 , srv661673521:89 , srv75680972:89 , srv1223086913:90 , srv812080159:90 , srv924035170:90 , srv980495516:90 , srv1603758642:91 , srv100837275:92 , srv1456614271:92 , srv283429645:93 , srv1375147799:516 }
2023-07-12 13:48:49,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:49,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962093843=46, srv222609149=58, srv398662393=71, srv1521855963=18, srv642241938=80, srv957321778=96, srv493715296=77, srv2095691565=50, srv706399119=83, srv1841562062=40, srv1193237605=6, srv484958044=76, srv1701534496=29, srv2056536767=49, srv283429645=64, srv229783671=59, srv249948106=62, srv1876694353=41, srv308326321=66, srv1829135175=39, srv1603758642=24, srv1699962470=28, srv880169726=91, srv1961713763=45, srv919560268=93, srv1456614271=16, srv2146295160=57, srv1902113792=42, srv2129205567=55, srv661673521=82, srv2106922214=52, srv1035756050=2, srv340578733=67, srv1101677117=3, srv236774952=60, srv257953247=63, srv456519577=73, srv624152668=79, srv1191235578=5, srv2123514174=54, srv951304711=95, srv845466672=89, srv1667534369=26, srv1771287619=34, srv1723705619=32, srv650040947=81, srv715342987=84, srv1290815854=10, srv2113325377=53, srv1939351619=44, srv1531247696=20, srv177958343=35, srv1175575492=4, srv1589406299=23, srv480579889=75, srv812080159=88, srv30542737=65, srv732699301=85, srv1745030600=33, srv393558866=70, srv480392790=74, srv1781807016=36, srv865899833=90, srv1795959064=38, srv924035170=94, srv247536349=61, srv768338738=87, srv2033003391=48, srv372496195=69, srv1789020538=37, srv1556118384=21, srv41382483=72, srv131952737=11, srv1223086913=8, srv1933660610=43, srv2134279539=56, srv1472380787=17, srv1564836788=22, srv1198279832=7, srv980495516=99, srv960204052=97, srv2097661951=51, srv1620890955=25, srv1427997487=15, srv169488281=27, srv1709017904=31, srv170713757=30, srv1035731114=1, srv136426960=12, srv199349859=47, srv75680972=86, srv1528553455=19, srv358730471=68, srv978897302=98, srv509255728=78, srv1375147799=13, srv904513412=92, srv1289097621=9, srv100837275=0, srv1392568048=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:49,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:49,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:49,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:49,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:49,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:49,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:49,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:49,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:49,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 13:48:49,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 13:48:49,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 13:48:49,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 13:48:49,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 13:48:49,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 13:48:49,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 13:48:49,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 13:48:49,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 13:48:49,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 13:48:50,013 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018817783906348487); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02979541163389459, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018817783906348487); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,014 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 13:48:50,014 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014651685049171689, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018817783906348487); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02979541163389459, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018817783906348487); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 13:48:50,231 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 313 ms to try 5000 different iterations.  Found a solution that moves 993 regions; Going from a computed imbalance of 0.014651685049171689 to a new imbalance of 5.049563947137419E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004650122992552569); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007324451309205374); MoveCostFunction : (multiplier=7.0, imbalance=0.11033333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004650122992552569); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,234 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1531247696:81 , srv283429645:81 , srv1035731114:82 , srv136426960:82 , srv2056536767:82 , srv480579889:82 , srv661673521:82 , srv812080159:82 , srv1603758642:83 , srv75680972:83 , srv880169726:83 , srv100837275:84 , srv1191235578:84 , srv1223086913:84 , srv1289097621:84 , srv1589406299:84 , srv1667534369:84 , srv1699962470:84 , srv177958343:84 , srv1902113792:84 , srv1961713763:84 , srv1962093843:84 , srv2033003391:84 , srv30542737:84 , srv398662393:84 , srv960204052:84 , srv1290815854:85 , srv1392568048:85 , srv1427997487:85 , srv1456614271:85 , srv1472380787:85 , srv1556118384:85 , srv169488281:85 , srv170713757:85 , srv1841562062:85 , srv1876694353:85 , srv2113325377:85 , srv2129205567:85 , srv2146295160:85 , srv229783671:85 , srv247536349:85 , srv358730471:85 , srv372496195:85 , srv393558866:85 , srv41382483:85 , srv480392790:85 , srv484958044:85 , srv509255728:85 , srv768338738:85 , srv904513412:85 , srv951304711:85 , srv957321778:85 , srv978897302:85 , srv1035756050:86 , srv1101677117:86 , srv1193237605:86 , srv1528553455:86 , srv1564836788:86 , srv1709017904:86 , srv1745030600:86 , srv1771287619:86 , srv1795959064:86 , srv1933660610:86 , srv1939351619:86 , srv199349859:86 , srv2097661951:86 , srv2106922214:86 , srv2123514174:86 , srv2134279539:86 , srv222609149:86 , srv249948106:86 , srv493715296:86 , srv706399119:86 , srv715342987:86 , srv732699301:86 , srv845466672:86 , srv924035170:86 , srv980495516:86 , srv1198279832:87 , srv131952737:87 , srv1521855963:87 , srv1620890955:87 , srv1723705619:87 , srv1781807016:87 , srv1789020538:87 , srv236774952:87 , srv257953247:87 , srv340578733:87 , srv456519577:87 , srv624152668:87 , srv642241938:87 , srv650040947:87 , srv865899833:87 , srv1175575492:88 , srv1701534496:88 , srv2095691565:88 , srv308326321:88 , srv919560268:88 , srv1829135175:89 , srv1375147799:558 }
2023-07-12 13:48:50,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:50,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962093843=46, srv222609149=58, srv398662393=71, srv1521855963=18, srv642241938=80, srv957321778=96, srv493715296=77, srv2095691565=50, srv706399119=83, srv1841562062=40, srv1193237605=6, srv484958044=76, srv1701534496=29, srv2056536767=49, srv283429645=64, srv229783671=59, srv249948106=62, srv1876694353=41, srv308326321=66, srv1829135175=39, srv1603758642=24, srv1699962470=28, srv880169726=91, srv1961713763=45, srv919560268=93, srv1456614271=16, srv2146295160=57, srv1902113792=42, srv2129205567=55, srv661673521=82, srv2106922214=52, srv1035756050=2, srv340578733=67, srv1101677117=3, srv236774952=60, srv257953247=63, srv456519577=73, srv624152668=79, srv1191235578=5, srv2123514174=54, srv951304711=95, srv845466672=89, srv1667534369=26, srv1771287619=34, srv1723705619=32, srv650040947=81, srv715342987=84, srv1290815854=10, srv2113325377=53, srv1939351619=44, srv1531247696=20, srv177958343=35, srv1175575492=4, srv1589406299=23, srv480579889=75, srv812080159=88, srv30542737=65, srv732699301=85, srv1745030600=33, srv393558866=70, srv480392790=74, srv1781807016=36, srv865899833=90, srv1795959064=38, srv924035170=94, srv247536349=61, srv768338738=87, srv2033003391=48, srv372496195=69, srv1789020538=37, srv1556118384=21, srv41382483=72, srv131952737=11, srv1223086913=8, srv1933660610=43, srv2134279539=56, srv1472380787=17, srv1564836788=22, srv1198279832=7, srv980495516=99, srv960204052=97, srv2097661951=51, srv1620890955=25, srv1427997487=15, srv169488281=27, srv1709017904=31, srv170713757=30, srv1035731114=1, srv136426960=12, srv199349859=47, srv75680972=86, srv1528553455=19, srv358730471=68, srv978897302=98, srv509255728=78, srv1375147799=13, srv904513412=92, srv1289097621=9, srv100837275=0, srv1392568048=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:50,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:50,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:50,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:50,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:50,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:50,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:50,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 13:48:50,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 13:48:50,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 13:48:50,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 13:48:50,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 13:48:50,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 13:48:50,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 13:48:50,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 13:48:50,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 13:48:50,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004650122992552569); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007324451309205374); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004650122992552569); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,360 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 13:48:50,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.980032601014159E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004650122992552569); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007324451309205374); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004650122992552569); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 13:48:50,625 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 345 ms to try 5000 different iterations.  Found a solution that moves 222 regions; Going from a computed imbalance of 4.980032601014159E-4 to a new imbalance of 1.9450630916641795E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.024666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,625 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1101677117:83 , srv1521855963:83 , srv1620890955:83 , srv170713757:83 , srv1829135175:83 , srv398662393:83 , srv865899833:83 , srv100837275:84 , srv1035731114:84 , srv1035756050:84 , srv1193237605:84 , srv1198279832:84 , srv1289097621:84 , srv131952737:84 , srv136426960:84 , srv1528553455:84 , srv1556118384:84 , srv1589406299:84 , srv1701534496:84 , srv1709017904:84 , srv1723705619:84 , srv1781807016:84 , srv1795959064:84 , srv1841562062:84 , srv1876694353:84 , srv1933660610:84 , srv1962093843:84 , srv199349859:84 , srv2056536767:84 , srv2097661951:84 , srv2129205567:84 , srv2134279539:84 , srv247536349:84 , srv30542737:84 , srv308326321:84 , srv340578733:84 , srv358730471:84 , srv393558866:84 , srv41382483:84 , srv493715296:84 , srv509255728:84 , srv624152668:84 , srv642241938:84 , srv650040947:84 , srv706399119:84 , srv715342987:84 , srv732699301:84 , srv845466672:84 , srv919560268:84 , srv924035170:84 , srv957321778:84 , srv980495516:84 , srv1175575492:85 , srv1191235578:85 , srv1223086913:85 , srv1290815854:85 , srv1392568048:85 , srv1427997487:85 , srv1456614271:85 , srv1472380787:85 , srv1531247696:85 , srv1564836788:85 , srv1603758642:85 , srv1667534369:85 , srv169488281:85 , srv1699962470:85 , srv1745030600:85 , srv1771287619:85 , srv177958343:85 , srv1789020538:85 , srv1902113792:85 , srv1939351619:85 , srv1961713763:85 , srv2033003391:85 , srv2095691565:85 , srv2106922214:85 , srv2113325377:85 , srv2123514174:85 , srv222609149:85 , srv229783671:85 , srv236774952:85 , srv249948106:85 , srv257953247:85 , srv283429645:85 , srv372496195:85 , srv456519577:85 , srv480392790:85 , srv480579889:85 , srv484958044:85 , srv661673521:85 , srv75680972:85 , srv768338738:85 , srv812080159:85 , srv880169726:85 , srv904513412:85 , srv951304711:85 , srv960204052:85 , srv978897302:85 , srv2146295160:86 , srv1375147799:643 }
2023-07-12 13:48:50,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:50,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962093843=46, srv222609149=58, srv398662393=71, srv1521855963=18, srv642241938=80, srv957321778=96, srv493715296=77, srv2095691565=50, srv706399119=83, srv1841562062=40, srv1193237605=6, srv484958044=76, srv1701534496=29, srv2056536767=49, srv283429645=64, srv229783671=59, srv249948106=62, srv1876694353=41, srv308326321=66, srv1829135175=39, srv1603758642=24, srv1699962470=28, srv880169726=91, srv1961713763=45, srv919560268=93, srv1456614271=16, srv2146295160=57, srv1902113792=42, srv2129205567=55, srv661673521=82, srv2106922214=52, srv1035756050=2, srv340578733=67, srv1101677117=3, srv236774952=60, srv257953247=63, srv456519577=73, srv624152668=79, srv1191235578=5, srv2123514174=54, srv951304711=95, srv845466672=89, srv1667534369=26, srv1771287619=34, srv1723705619=32, srv650040947=81, srv715342987=84, srv1290815854=10, srv2113325377=53, srv1939351619=44, srv1531247696=20, srv177958343=35, srv1175575492=4, srv1589406299=23, srv480579889=75, srv812080159=88, srv30542737=65, srv732699301=85, srv1745030600=33, srv393558866=70, srv480392790=74, srv1781807016=36, srv865899833=90, srv1795959064=38, srv924035170=94, srv247536349=61, srv768338738=87, srv2033003391=48, srv372496195=69, srv1789020538=37, srv1556118384=21, srv41382483=72, srv131952737=11, srv1223086913=8, srv1933660610=43, srv2134279539=56, srv1472380787=17, srv1564836788=22, srv1198279832=7, srv980495516=99, srv960204052=97, srv2097661951=51, srv1620890955=25, srv1427997487=15, srv169488281=27, srv1709017904=31, srv170713757=30, srv1035731114=1, srv136426960=12, srv199349859=47, srv75680972=86, srv1528553455=19, srv358730471=68, srv978897302=98, srv509255728=78, srv1375147799=13, srv904513412=92, srv1289097621=9, srv100837275=0, srv1392568048=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:50,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:50,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:50,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 13:48:50,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 13:48:50,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 13:48:50,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 13:48:50,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 13:48:50,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 13:48:50,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 13:48:50,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 13:48:50,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 13:48:50,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 13:48:50,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 13:48:50,752 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.789615369213387E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,753 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1101677117:83 , srv1521855963:83 , srv1620890955:83 , srv170713757:83 , srv1829135175:83 , srv398662393:83 , srv865899833:83 , srv100837275:84 , srv1035731114:84 , srv1035756050:84 , srv1193237605:84 , srv1198279832:84 , srv1289097621:84 , srv131952737:84 , srv136426960:84 , srv1528553455:84 , srv1556118384:84 , srv1589406299:84 , srv1701534496:84 , srv1709017904:84 , srv1723705619:84 , srv1781807016:84 , srv1795959064:84 , srv1841562062:84 , srv1876694353:84 , srv1933660610:84 , srv1962093843:84 , srv199349859:84 , srv2056536767:84 , srv2097661951:84 , srv2129205567:84 , srv2134279539:84 , srv247536349:84 , srv30542737:84 , srv308326321:84 , srv340578733:84 , srv358730471:84 , srv393558866:84 , srv41382483:84 , srv493715296:84 , srv509255728:84 , srv624152668:84 , srv642241938:84 , srv650040947:84 , srv706399119:84 , srv715342987:84 , srv732699301:84 , srv845466672:84 , srv919560268:84 , srv924035170:84 , srv957321778:84 , srv980495516:84 , srv1175575492:85 , srv1191235578:85 , srv1223086913:85 , srv1290815854:85 , srv1392568048:85 , srv1427997487:85 , srv1456614271:85 , srv1472380787:85 , srv1531247696:85 , srv1564836788:85 , srv1603758642:85 , srv1667534369:85 , srv169488281:85 , srv1699962470:85 , srv1745030600:85 , srv1771287619:85 , srv177958343:85 , srv1789020538:85 , srv1902113792:85 , srv1939351619:85 , srv1961713763:85 , srv2033003391:85 , srv2095691565:85 , srv2106922214:85 , srv2113325377:85 , srv2123514174:85 , srv222609149:85 , srv229783671:85 , srv236774952:85 , srv249948106:85 , srv257953247:85 , srv283429645:85 , srv372496195:85 , srv456519577:85 , srv480392790:85 , srv480579889:85 , srv484958044:85 , srv661673521:85 , srv75680972:85 , srv768338738:85 , srv812080159:85 , srv880169726:85 , srv904513412:85 , srv951304711:85 , srv960204052:85 , srv978897302:85 , srv2146295160:86 , srv1375147799:643 }
2023-07-12 13:48:50,783 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=438 (was 434) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=3285 (was 3785)
2023-07-12 13:48:50,791 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=438, ProcessCount=174, AvailableMemoryMB=3285
2023-07-12 13:48:50,792 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 13:48:50,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-12 13:48:50,804 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1079610390:84 , srv1204792320:84 , srv1296689772:84 , srv1444310692:84 , srv1597992213:84 , srv1681564555:84 , srv171606069:84 , srv1748133783:84 , srv1807442056:84 , srv1917961011:84 , srv2061243512:84 , srv2121465970:84 , srv214482533:84 , srv226974714:84 , srv259004419:84 , srv313119469:84 , srv332260864:84 , srv366734880:84 , srv398014724:84 , srv413591458:84 , srv501417457:84 , srv522136122:84 , srv610905911:84 , srv688363007:84 , srv728415375:84 , srv802058341:84 , srv837157628:84 , srv929841078:84 , srv974332010:84 , srv2092680309:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-12 13:48:50,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:48:50,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv688363007=24, srv259004419=15, srv171606069=6, srv214482533=13, srv366734880=18, srv1807442056=8, srv728415375=25, srv1597992213=4, srv2061243512=10, srv332260864=17, srv1079610390=0, srv398014724=19, srv2121465970=12, srv929841078=28, srv522136122=22, srv1204792320=1, srv1748133783=7, srv413591458=20, srv1681564555=5, srv837157628=27, srv2092680309=11, srv1296689772=2, srv501417457=21, srv226974714=14, srv610905911=23, srv1917961011=9, srv974332010=29, srv802058341=26, srv1444310692=3, srv313119469=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:48:50,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:48:50,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:48:50,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:48:50,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-12 13:48:50,847 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.144448145238046, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:48:50,847 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912648194653259, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.144448145238046, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-12 13:50:50,827 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 8156383 different iterations.  Found a solution that moves 1848 regions; Going from a computed imbalance of 0.9912648194653259 to a new imbalance of 4.313324190526491E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:50:50,833 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1079610390:90 , srv1204792320:90 , srv1296689772:90 , srv1444310692:90 , srv1597992213:90 , srv1681564555:90 , srv171606069:90 , srv1748133783:90 , srv1807442056:90 , srv1917961011:90 , srv2061243512:90 , srv2092680309:90 , srv2121465970:90 , srv214482533:90 , srv226974714:90 , srv259004419:90 , srv313119469:90 , srv332260864:90 , srv366734880:90 , srv398014724:90 , srv413591458:90 , srv501417457:90 , srv522136122:90 , srv610905911:90 , srv688363007:90 , srv728415375:90 , srv802058341:90 , srv837157628:90 , srv929841078:90 , srv974332010:90 }
2023-07-12 13:50:50,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 13:50:50,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv688363007=24, srv259004419=15, srv171606069=6, srv214482533=13, srv366734880=18, srv1807442056=8, srv728415375=25, srv1597992213=4, srv2061243512=10, srv332260864=17, srv1079610390=0, srv398014724=19, srv2121465970=12, srv929841078=28, srv522136122=22, srv1204792320=1, srv1748133783=7, srv413591458=20, srv1681564555=5, srv837157628=27, srv2092680309=11, srv1296689772=2, srv501417457=21, srv226974714=14, srv610905911=23, srv1917961011=9, srv974332010=29, srv802058341=26, srv1444310692=3, srv313119469=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 13:50:50,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 13:50:50,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-12 13:50:50,848 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 13:50:50,848 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1079610390:90 , srv1204792320:90 , srv1296689772:90 , srv1444310692:90 , srv1597992213:90 , srv1681564555:90 , srv171606069:90 , srv1748133783:90 , srv1807442056:90 , srv1917961011:90 , srv2061243512:90 , srv2092680309:90 , srv2121465970:90 , srv214482533:90 , srv226974714:90 , srv259004419:90 , srv313119469:90 , srv332260864:90 , srv366734880:90 , srv398014724:90 , srv413591458:90 , srv501417457:90 , srv522136122:90 , srv610905911:90 , srv688363007:90 , srv728415375:90 , srv802058341:90 , srv837157628:90 , srv929841078:90 , srv974332010:90 }
2023-07-12 13:50:50,860 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=413 (was 438), ProcessCount=176 (was 174) - ProcessCount LEAK? -, AvailableMemoryMB=2750 (was 3285)