2023-04-05 11:56:49,341 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-05 11:56:49,658 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-05 11:56:49,769 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-05 11:56:49,828 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-05 11:56:49,835 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-05 11:56:49,848 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=342, ProcessCount=169, AvailableMemoryMB=2060
2023-04-05 11:56:49,850 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-05 11:56:49,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-05 11:56:49,964 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1023515827:84 , srv1023599614:84 , srv1049646457:84 , srv1059369145:84 , srv1063028487:84 , srv1104936203:84 , srv1110138019:84 , srv1133616251:84 , srv1158335950:84 , srv1171792683:84 , srv1210050013:84 , srv1217421301:84 , srv1244484004:84 , srv1248926872:84 , srv1260500133:84 , srv1272259358:84 , srv1298038176:84 , srv1319904968:84 , srv1339387241:84 , srv141075908:84 , srv1431377630:84 , srv1436739557:84 , srv1525870338:84 , srv153688316:84 , srv154736627:84 , srv1598895408:84 , srv1610898503:84 , srv1646886194:84 , srv1666680931:84 , srv1696863187:84 , srv1718062600:84 , srv1742984698:84 , srv1778292901:84 , srv1792159555:84 , srv1793433010:84 , srv1862347536:84 , srv1866088982:84 , srv1902812087:84 , srv1909508939:84 , srv1928369242:84 , srv1947156373:84 , srv1955065516:84 , srv1963395528:84 , srv1982092511:84 , srv1982314573:84 , srv1992970275:84 , srv2025742693:84 , srv2042964727:84 , srv2101332616:84 , srv2110775390:84 , srv2112140628:84 , srv2115390995:84 , srv2117798353:84 , srv2134586872:84 , srv214812254:84 , srv221349220:84 , srv226115022:84 , srv226569562:84 , srv270946925:84 , srv292081656:84 , srv301621262:84 , srv30202984:84 , srv308140820:84 , srv313537313:84 , srv342903951:84 , srv360996310:84 , srv375757552:84 , srv407521046:84 , srv429452724:84 , srv452026854:84 , srv452141908:84 , srv470184327:84 , srv478964925:84 , srv483394672:84 , srv484009836:84 , srv514174681:84 , srv523173461:84 , srv535139519:84 , srv561321044:84 , srv580142951:84 , srv600967254:84 , srv605337130:84 , srv640607967:84 , srv674308137:84 , srv714606366:84 , srv717579028:84 , srv724119788:84 , srv73671979:84 , srv761721369:84 , srv765789610:84 , srv812884436:84 , srv816331579:84 , srv839078906:84 , srv862659693:84 , srv86658847:84 , srv878632611:84 , srv890402130:84 , srv895118572:84 , srv905415131:84 , srv1653219103:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-05 11:56:49,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv714606366=85, srv862659693=94, srv429452724=69, srv292081656=60, srv1982092511=44, srv1718062600=31, srv478964925=73, srv1992970275=46, srv812884436=91, srv1902812087=38, srv1210050013=10, srv1158335950=8, srv313537313=64, srv1110138019=6, srv761721369=89, srv1653219103=28, srv2025742693=47, srv153688316=23, srv2112140628=51, srv1063028487=4, srv1059369145=3, srv1955065516=42, srv2115390995=52, srv1928369242=40, srv452141908=71, srv1666680931=29, srv30202984=62, srv1598895408=25, srv2134586872=54, srv214812254=55, srv226115022=57, srv308140820=63, srv1298038176=16, srv561321044=79, srv717579028=86, srv1742984698=32, srv375757552=67, srv895118572=98, srv470184327=72, srv360996310=66, srv73671979=88, srv1862347536=36, srv452026854=70, srv141075908=19, srv514174681=76, srv1525870338=22, srv1260500133=14, srv890402130=97, srv1319904968=17, srv1866088982=37, srv1431377630=20, srv1792159555=34, srv580142951=80, srv2110775390=50, srv905415131=99, srv1133616251=7, srv2042964727=48, srv1104936203=5, srv2101332616=49, srv1272259358=15, srv1610898503=26, srv1436739557=21, srv1248926872=13, srv765789610=90, srv1947156373=41, srv523173461=77, srv1646886194=27, srv1778292901=33, srv878632611=96, srv1696863187=30, srv2117798353=53, srv1909508939=39, srv226569562=58, srv1023599614=1, srv270946925=59, srv605337130=82, srv407521046=68, srv1023515827=0, srv640607967=83, srv1171792683=9, srv484009836=75, srv86658847=95, srv535139519=78, srv483394672=74, srv1049646457=2, srv1339387241=18, srv301621262=61, srv600967254=81, srv816331579=92, srv1244484004=12, srv1963395528=43, srv342903951=65, srv154736627=24, srv1982314573=45, srv1217421301=11, srv674308137=84, srv1793433010=35, srv221349220=56, srv839078906=93, srv724119788=87} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:50,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:50,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:50,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-05 11:56:50,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-05 11:56:50,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-05 11:56:50,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-05 11:56:50,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-05 11:56:50,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-05 11:56:50,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-05 11:56:50,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-05 11:56:50,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:50,227 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-05 11:56:50,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-05 11:56:50,876 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 787 ms to try 5000 different iterations.  Found a solution that moves 4674 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17472974459035406. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041035967361376414, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06073252798218284, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5193333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041035967361376414, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16983333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23691666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:50,898 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1023599614:81 , srv360996310:82 , srv153688316:83 , srv154736627:83 , srv214812254:83 , srv292081656:83 , srv483394672:83 , srv1272259358:84 , srv1598895408:84 , srv1610898503:84 , srv1792159555:84 , srv226569562:84 , srv523173461:84 , srv724119788:84 , srv890402130:84 , srv895118572:84 , srv905415131:84 , srv1210050013:85 , srv1298038176:85 , srv141075908:85 , srv1866088982:85 , srv1909508939:85 , srv1928369242:85 , srv1947156373:85 , srv1963395528:85 , srv2115390995:85 , srv2134586872:85 , srv221349220:85 , srv452026854:85 , srv514174681:85 , srv761721369:85 , srv812884436:85 , srv862659693:85 , srv878632611:85 , srv1023515827:86 , srv1049646457:86 , srv1104936203:86 , srv1110138019:86 , srv1133616251:86 , srv1171792683:86 , srv1339387241:86 , srv1431377630:86 , srv1525870338:86 , srv1742984698:86 , srv1982092511:86 , srv1992970275:86 , srv2110775390:86 , srv2112140628:86 , srv301621262:86 , srv30202984:86 , srv308140820:86 , srv429452724:86 , srv484009836:86 , srv535139519:86 , srv561321044:86 , srv674308137:86 , srv714606366:86 , srv839078906:86 , srv1158335950:87 , srv1244484004:87 , srv1260500133:87 , srv1436739557:87 , srv1646886194:87 , srv1696863187:87 , srv1793433010:87 , srv1955065516:87 , srv1982314573:87 , srv2025742693:87 , srv342903951:87 , srv452141908:87 , srv605337130:87 , srv717579028:87 , srv816331579:87 , srv86658847:87 , srv1063028487:88 , srv1248926872:88 , srv1718062600:88 , srv1778292901:88 , srv1862347536:88 , srv1902812087:88 , srv2117798353:88 , srv313537313:88 , srv375757552:88 , srv407521046:88 , srv580142951:88 , srv600967254:88 , srv73671979:88 , srv765789610:88 , srv1059369145:89 , srv1666680931:89 , srv2101332616:89 , srv226115022:89 , srv478964925:89 , srv640607967:89 , srv270946925:90 , srv470184327:90 , srv2042964727:91 , srv1217421301:93 , srv1319904968:96 , srv1653219103:455 }
2023-04-05 11:56:50,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:50,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv714606366=85, srv862659693=94, srv429452724=69, srv292081656=60, srv1982092511=44, srv1718062600=31, srv478964925=73, srv1992970275=46, srv812884436=91, srv1902812087=38, srv1210050013=10, srv1158335950=8, srv313537313=64, srv1110138019=6, srv761721369=89, srv1653219103=28, srv2025742693=47, srv153688316=23, srv2112140628=51, srv1063028487=4, srv1059369145=3, srv1955065516=42, srv2115390995=52, srv1928369242=40, srv452141908=71, srv1666680931=29, srv30202984=62, srv1598895408=25, srv2134586872=54, srv214812254=55, srv226115022=57, srv308140820=63, srv1298038176=16, srv561321044=79, srv717579028=86, srv1742984698=32, srv375757552=67, srv895118572=98, srv470184327=72, srv360996310=66, srv73671979=88, srv1862347536=36, srv452026854=70, srv141075908=19, srv514174681=76, srv1525870338=22, srv1260500133=14, srv890402130=97, srv1319904968=17, srv1866088982=37, srv1431377630=20, srv1792159555=34, srv580142951=80, srv2110775390=50, srv905415131=99, srv1133616251=7, srv2042964727=48, srv1104936203=5, srv2101332616=49, srv1272259358=15, srv1610898503=26, srv1436739557=21, srv1248926872=13, srv765789610=90, srv1947156373=41, srv523173461=77, srv1646886194=27, srv1778292901=33, srv878632611=96, srv1696863187=30, srv2117798353=53, srv1909508939=39, srv226569562=58, srv1023599614=1, srv270946925=59, srv605337130=82, srv407521046=68, srv1023515827=0, srv640607967=83, srv1171792683=9, srv484009836=75, srv86658847=95, srv535139519=78, srv483394672=74, srv1049646457=2, srv1339387241=18, srv301621262=61, srv600967254=81, srv816331579=92, srv1244484004=12, srv1963395528=43, srv342903951=65, srv154736627=24, srv1982314573=45, srv1217421301=11, srv674308137=84, srv1793433010=35, srv221349220=56, srv839078906=93, srv724119788=87} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:50,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:50,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:50,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-05 11:56:50,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-05 11:56:50,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-05 11:56:50,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-05 11:56:50,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-05 11:56:50,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-05 11:56:50,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-05 11:56:51,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041035967361376414, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06073252798218284, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041035967361376414, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16983333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23691666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:51,067 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-05 11:56:51,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1746970165428435, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041035967361376414, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06073252798218284, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041035967361376414, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16983333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23691666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-05 11:56:51,372 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 429 ms to try 5000 different iterations.  Found a solution that moves 2625 regions; Going from a computed imbalance of 0.1746970165428435 to a new imbalance of 0.013921597785187732. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100039692123985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2916666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:51,381 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1319904968:76 , srv1217421301:79 , srv270946925:79 , srv2042964727:80 , srv1902812087:81 , srv470184327:81 , srv1666680931:82 , srv2117798353:82 , srv226115022:82 , srv605337130:82 , srv1059369145:83 , srv1063028487:83 , srv1133616251:83 , srv1248926872:83 , srv1862347536:83 , srv2101332616:83 , srv452141908:83 , srv73671979:83 , srv1436739557:84 , srv1718062600:84 , srv1742984698:84 , srv1909508939:84 , srv2025742693:84 , srv301621262:84 , srv313537313:84 , srv429452724:84 , srv640607967:84 , srv714606366:84 , srv816331579:84 , srv1244484004:85 , srv1696863187:85 , srv1982092511:85 , srv342903951:85 , srv375757552:85 , srv407521046:85 , srv478964925:85 , srv484009836:85 , srv580142951:85 , srv724119788:85 , srv761721369:85 , srv1023515827:86 , srv1049646457:86 , srv1158335950:86 , srv1210050013:86 , srv1260500133:86 , srv1339387241:86 , srv1431377630:86 , srv154736627:86 , srv1646886194:86 , srv1778292901:86 , srv1793433010:86 , srv1866088982:86 , srv1982314573:86 , srv2112140628:86 , srv2115390995:86 , srv221349220:86 , srv226569562:86 , srv535139519:86 , srv561321044:86 , srv600967254:86 , srv717579028:86 , srv765789610:86 , srv839078906:86 , srv86658847:86 , srv1104936203:87 , srv1272259358:87 , srv141075908:87 , srv1525870338:87 , srv1792159555:87 , srv1947156373:87 , srv1963395528:87 , srv1992970275:87 , srv2110775390:87 , srv30202984:87 , srv514174681:87 , srv812884436:87 , srv878632611:87 , srv1110138019:88 , srv1171792683:88 , srv153688316:88 , srv1598895408:88 , srv1610898503:88 , srv1928369242:88 , srv1955065516:88 , srv452026854:88 , srv674308137:88 , srv890402130:88 , srv895118572:88 , srv1298038176:89 , srv2134586872:89 , srv214812254:89 , srv308140820:89 , srv862659693:89 , srv905415131:89 , srv1023599614:90 , srv523173461:90 , srv360996310:91 , srv292081656:92 , srv483394672:92 , srv1653219103:521 }
2023-04-05 11:56:51,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:51,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv714606366=85, srv862659693=94, srv429452724=69, srv292081656=60, srv1982092511=44, srv1718062600=31, srv478964925=73, srv1992970275=46, srv812884436=91, srv1902812087=38, srv1210050013=10, srv1158335950=8, srv313537313=64, srv1110138019=6, srv761721369=89, srv1653219103=28, srv2025742693=47, srv153688316=23, srv2112140628=51, srv1063028487=4, srv1059369145=3, srv1955065516=42, srv2115390995=52, srv1928369242=40, srv452141908=71, srv1666680931=29, srv30202984=62, srv1598895408=25, srv2134586872=54, srv214812254=55, srv226115022=57, srv308140820=63, srv1298038176=16, srv561321044=79, srv717579028=86, srv1742984698=32, srv375757552=67, srv895118572=98, srv470184327=72, srv360996310=66, srv73671979=88, srv1862347536=36, srv452026854=70, srv141075908=19, srv514174681=76, srv1525870338=22, srv1260500133=14, srv890402130=97, srv1319904968=17, srv1866088982=37, srv1431377630=20, srv1792159555=34, srv580142951=80, srv2110775390=50, srv905415131=99, srv1133616251=7, srv2042964727=48, srv1104936203=5, srv2101332616=49, srv1272259358=15, srv1610898503=26, srv1436739557=21, srv1248926872=13, srv765789610=90, srv1947156373=41, srv523173461=77, srv1646886194=27, srv1778292901=33, srv878632611=96, srv1696863187=30, srv2117798353=53, srv1909508939=39, srv226569562=58, srv1023599614=1, srv270946925=59, srv605337130=82, srv407521046=68, srv1023515827=0, srv640607967=83, srv1171792683=9, srv484009836=75, srv86658847=95, srv535139519=78, srv483394672=74, srv1049646457=2, srv1339387241=18, srv301621262=61, srv600967254=81, srv816331579=92, srv1244484004=12, srv1963395528=43, srv342903951=65, srv154736627=24, srv1982314573=45, srv1217421301=11, srv674308137=84, srv1793433010=35, srv221349220=56, srv839078906=93, srv724119788=87} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:51,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:51,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-05 11:56:51,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-05 11:56:51,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-05 11:56:51,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-05 11:56:51,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-05 11:56:51,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-05 11:56:51,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-05 11:56:51,544 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100039692123985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:51,544 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-05 11:56:51,544 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013903217142330374, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04100039692123985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-05 11:56:51,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 339 ms to try 5000 different iterations.  Found a solution that moves 931 regions; Going from a computed imbalance of 0.013903217142330374 to a new imbalance of 6.757520753367511E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012615348818550065); MoveCostFunction : (multiplier=7.0, imbalance=0.10344444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:51,769 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1955065516:82 , srv292081656:82 , srv308140820:82 , srv483394672:82 , srv1171792683:83 , srv1298038176:83 , srv1778292901:83 , srv2134586872:83 , srv30202984:83 , srv600967254:83 , srv765789610:83 , srv1110138019:84 , srv1260500133:84 , srv1793433010:84 , srv1928369242:84 , srv375757552:84 , srv407521046:84 , srv478964925:84 , srv523173461:84 , srv640607967:84 , srv674308137:84 , srv862659693:84 , srv86658847:84 , srv1049646457:85 , srv1059369145:85 , srv1104936203:85 , srv1158335950:85 , srv1217421301:85 , srv1244484004:85 , srv1339387241:85 , srv1431377630:85 , srv1525870338:85 , srv1598895408:85 , srv1646886194:85 , srv1718062600:85 , srv1963395528:85 , srv1982314573:85 , srv1992970275:85 , srv2101332616:85 , srv2110775390:85 , srv2112140628:85 , srv221349220:85 , srv313537313:85 , srv360996310:85 , srv452026854:85 , srv535139519:85 , srv561321044:85 , srv580142951:85 , srv717579028:85 , srv812884436:85 , srv839078906:85 , srv878632611:85 , srv905415131:85 , srv1023515827:86 , srv1023599614:86 , srv1272259358:86 , srv141075908:86 , srv1436739557:86 , srv153688316:86 , srv1610898503:86 , srv1696863187:86 , srv1792159555:86 , srv1982092511:86 , srv226115022:86 , srv342903951:86 , srv514174681:86 , srv73671979:86 , srv895118572:86 , srv1063028487:87 , srv1210050013:87 , srv1248926872:87 , srv1319904968:87 , srv1666680931:87 , srv1742984698:87 , srv1862347536:87 , srv1947156373:87 , srv2025742693:87 , srv2042964727:87 , srv2115390995:87 , srv214812254:87 , srv270946925:87 , srv301621262:87 , srv452141908:87 , srv470184327:87 , srv484009836:87 , srv714606366:87 , srv890402130:87 , srv1133616251:88 , srv154736627:88 , srv1866088982:88 , srv1902812087:88 , srv1909508939:88 , srv226569562:88 , srv429452724:88 , srv605337130:88 , srv761721369:88 , srv816331579:88 , srv2117798353:89 , srv724119788:89 , srv1653219103:532 }
2023-04-05 11:56:51,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:51,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv714606366=85, srv862659693=94, srv429452724=69, srv292081656=60, srv1982092511=44, srv1718062600=31, srv478964925=73, srv1992970275=46, srv812884436=91, srv1902812087=38, srv1210050013=10, srv1158335950=8, srv313537313=64, srv1110138019=6, srv761721369=89, srv1653219103=28, srv2025742693=47, srv153688316=23, srv2112140628=51, srv1063028487=4, srv1059369145=3, srv1955065516=42, srv2115390995=52, srv1928369242=40, srv452141908=71, srv1666680931=29, srv30202984=62, srv1598895408=25, srv2134586872=54, srv214812254=55, srv226115022=57, srv308140820=63, srv1298038176=16, srv561321044=79, srv717579028=86, srv1742984698=32, srv375757552=67, srv895118572=98, srv470184327=72, srv360996310=66, srv73671979=88, srv1862347536=36, srv452026854=70, srv141075908=19, srv514174681=76, srv1525870338=22, srv1260500133=14, srv890402130=97, srv1319904968=17, srv1866088982=37, srv1431377630=20, srv1792159555=34, srv580142951=80, srv2110775390=50, srv905415131=99, srv1133616251=7, srv2042964727=48, srv1104936203=5, srv2101332616=49, srv1272259358=15, srv1610898503=26, srv1436739557=21, srv1248926872=13, srv765789610=90, srv1947156373=41, srv523173461=77, srv1646886194=27, srv1778292901=33, srv878632611=96, srv1696863187=30, srv2117798353=53, srv1909508939=39, srv226569562=58, srv1023599614=1, srv270946925=59, srv605337130=82, srv407521046=68, srv1023515827=0, srv640607967=83, srv1171792683=9, srv484009836=75, srv86658847=95, srv535139519=78, srv483394672=74, srv1049646457=2, srv1339387241=18, srv301621262=61, srv600967254=81, srv816331579=92, srv1244484004=12, srv1963395528=43, srv342903951=65, srv154736627=24, srv1982314573=45, srv1217421301=11, srv674308137=84, srv1793433010=35, srv221349220=56, srv839078906=93, srv724119788=87} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:51,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:51,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-05 11:56:51,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-05 11:56:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-05 11:56:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-05 11:56:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-05 11:56:51,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-05 11:56:51,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-05 11:56:51,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-05 11:56:51,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-05 11:56:51,900 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012615348818550065); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:51,900 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-05 11:56:51,900 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.692330740033417E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012615348818550065); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-05 11:56:52,211 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 408 ms to try 5000 different iterations.  Found a solution that moves 231 regions; Going from a computed imbalance of 6.692330740033417E-4 to a new imbalance of 1.5821355504794836E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00259499648053841); MoveCostFunction : (multiplier=7.0, imbalance=0.025666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:52,212 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv816331579:82 , srv1319904968:83 , srv1666680931:83 , srv1866088982:83 , srv2115390995:83 , srv2117798353:83 , srv890402130:83 , srv1023515827:84 , srv1063028487:84 , srv1104936203:84 , srv1110138019:84 , srv1158335950:84 , srv1210050013:84 , srv1244484004:84 , srv1431377630:84 , srv1525870338:84 , srv1610898503:84 , srv1696863187:84 , srv1862347536:84 , srv1902812087:84 , srv1947156373:84 , srv1982314573:84 , srv1992970275:84 , srv2042964727:84 , srv2134586872:84 , srv214812254:84 , srv226569562:84 , srv292081656:84 , srv301621262:84 , srv342903951:84 , srv429452724:84 , srv452026854:84 , srv470184327:84 , srv478964925:84 , srv484009836:84 , srv514174681:84 , srv523173461:84 , srv580142951:84 , srv674308137:84 , srv724119788:84 , srv761721369:84 , srv862659693:84 , srv895118572:84 , srv905415131:84 , srv1023599614:85 , srv1049646457:85 , srv1059369145:85 , srv1133616251:85 , srv1171792683:85 , srv1217421301:85 , srv1248926872:85 , srv1260500133:85 , srv1272259358:85 , srv1298038176:85 , srv1339387241:85 , srv141075908:85 , srv1436739557:85 , srv153688316:85 , srv154736627:85 , srv1598895408:85 , srv1646886194:85 , srv1718062600:85 , srv1742984698:85 , srv1792159555:85 , srv1793433010:85 , srv1909508939:85 , srv1928369242:85 , srv1955065516:85 , srv1963395528:85 , srv1982092511:85 , srv2025742693:85 , srv2101332616:85 , srv2110775390:85 , srv2112140628:85 , srv221349220:85 , srv226115022:85 , srv308140820:85 , srv313537313:85 , srv360996310:85 , srv375757552:85 , srv407521046:85 , srv452141908:85 , srv483394672:85 , srv535139519:85 , srv561321044:85 , srv600967254:85 , srv605337130:85 , srv640607967:85 , srv714606366:85 , srv717579028:85 , srv73671979:85 , srv765789610:85 , srv839078906:85 , srv86658847:85 , srv878632611:85 , srv1778292901:86 , srv270946925:86 , srv30202984:86 , srv812884436:86 , srv1653219103:633 }
2023-04-05 11:56:52,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:52,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv714606366=85, srv862659693=94, srv429452724=69, srv292081656=60, srv1982092511=44, srv1718062600=31, srv478964925=73, srv1992970275=46, srv812884436=91, srv1902812087=38, srv1210050013=10, srv1158335950=8, srv313537313=64, srv1110138019=6, srv761721369=89, srv1653219103=28, srv2025742693=47, srv153688316=23, srv2112140628=51, srv1063028487=4, srv1059369145=3, srv1955065516=42, srv2115390995=52, srv1928369242=40, srv452141908=71, srv1666680931=29, srv30202984=62, srv1598895408=25, srv2134586872=54, srv214812254=55, srv226115022=57, srv308140820=63, srv1298038176=16, srv561321044=79, srv717579028=86, srv1742984698=32, srv375757552=67, srv895118572=98, srv470184327=72, srv360996310=66, srv73671979=88, srv1862347536=36, srv452026854=70, srv141075908=19, srv514174681=76, srv1525870338=22, srv1260500133=14, srv890402130=97, srv1319904968=17, srv1866088982=37, srv1431377630=20, srv1792159555=34, srv580142951=80, srv2110775390=50, srv905415131=99, srv1133616251=7, srv2042964727=48, srv1104936203=5, srv2101332616=49, srv1272259358=15, srv1610898503=26, srv1436739557=21, srv1248926872=13, srv765789610=90, srv1947156373=41, srv523173461=77, srv1646886194=27, srv1778292901=33, srv878632611=96, srv1696863187=30, srv2117798353=53, srv1909508939=39, srv226569562=58, srv1023599614=1, srv270946925=59, srv605337130=82, srv407521046=68, srv1023515827=0, srv640607967=83, srv1171792683=9, srv484009836=75, srv86658847=95, srv535139519=78, srv483394672=74, srv1049646457=2, srv1339387241=18, srv301621262=61, srv600967254=81, srv816331579=92, srv1244484004=12, srv1963395528=43, srv342903951=65, srv154736627=24, srv1982314573=45, srv1217421301=11, srv674308137=84, srv1793433010=35, srv221349220=56, srv839078906=93, srv724119788=87} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:52,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:52,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-05 11:56:52,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-05 11:56:52,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-05 11:56:52,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-05 11:56:52,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-05 11:56:52,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-05 11:56:52,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-05 11:56:52,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-05 11:56:52,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-05 11:56:52,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-05 11:56:52,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.4203858933347402E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00259499648053841); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:52,338 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv816331579:82 , srv1319904968:83 , srv1666680931:83 , srv1866088982:83 , srv2115390995:83 , srv2117798353:83 , srv890402130:83 , srv1023515827:84 , srv1063028487:84 , srv1104936203:84 , srv1110138019:84 , srv1158335950:84 , srv1210050013:84 , srv1244484004:84 , srv1431377630:84 , srv1525870338:84 , srv1610898503:84 , srv1696863187:84 , srv1862347536:84 , srv1902812087:84 , srv1947156373:84 , srv1982314573:84 , srv1992970275:84 , srv2042964727:84 , srv2134586872:84 , srv214812254:84 , srv226569562:84 , srv292081656:84 , srv301621262:84 , srv342903951:84 , srv429452724:84 , srv452026854:84 , srv470184327:84 , srv478964925:84 , srv484009836:84 , srv514174681:84 , srv523173461:84 , srv580142951:84 , srv674308137:84 , srv724119788:84 , srv761721369:84 , srv862659693:84 , srv895118572:84 , srv905415131:84 , srv1023599614:85 , srv1049646457:85 , srv1059369145:85 , srv1133616251:85 , srv1171792683:85 , srv1217421301:85 , srv1248926872:85 , srv1260500133:85 , srv1272259358:85 , srv1298038176:85 , srv1339387241:85 , srv141075908:85 , srv1436739557:85 , srv153688316:85 , srv154736627:85 , srv1598895408:85 , srv1646886194:85 , srv1718062600:85 , srv1742984698:85 , srv1792159555:85 , srv1793433010:85 , srv1909508939:85 , srv1928369242:85 , srv1955065516:85 , srv1963395528:85 , srv1982092511:85 , srv2025742693:85 , srv2101332616:85 , srv2110775390:85 , srv2112140628:85 , srv221349220:85 , srv226115022:85 , srv308140820:85 , srv313537313:85 , srv360996310:85 , srv375757552:85 , srv407521046:85 , srv452141908:85 , srv483394672:85 , srv535139519:85 , srv561321044:85 , srv600967254:85 , srv605337130:85 , srv640607967:85 , srv714606366:85 , srv717579028:85 , srv73671979:85 , srv765789610:85 , srv839078906:85 , srv86658847:85 , srv878632611:85 , srv1778292901:86 , srv270946925:86 , srv30202984:86 , srv812884436:86 , srv1653219103:633 }
2023-04-05 11:56:52,363 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=342 (was 342), ProcessCount=169 (was 169), AvailableMemoryMB=1571 (was 2060)
2023-04-05 11:56:52,370 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=342, ProcessCount=169, AvailableMemoryMB=1571
2023-04-05 11:56:52,371 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-05 11:56:52,371 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-05 11:56:52,382 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1085298644:84 , srv1153760924:84 , srv1175380324:84 , srv1183703484:84 , srv1267018069:84 , srv127783693:84 , srv136269792:84 , srv1412949667:84 , srv1487656786:84 , srv1516392017:84 , srv1595369852:84 , srv1598980863:84 , srv1665594810:84 , srv174401704:84 , srv1795715519:84 , srv1817609365:84 , srv1898232070:84 , srv1989089832:84 , srv270100271:84 , srv355761467:84 , srv485554897:84 , srv539185824:84 , srv539228648:84 , srv599885688:84 , srv661200055:84 , srv750362496:84 , srv824655252:84 , srv878894629:84 , srv961186815:84 , srv278466873:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-05 11:56:52,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:56:52,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv824655252=27, srv539185824=22, srv1183703484=3, srv1817609365=15, srv1795715519=14, srv1516392017=9, srv1412949667=7, srv136269792=6, srv599885688=24, srv1665594810=12, srv1595369852=10, srv1598980863=11, srv1175380324=2, srv1085298644=0, srv485554897=21, srv174401704=13, srv355761467=20, srv1267018069=4, srv127783693=5, srv1989089832=17, srv1487656786=8, srv750362496=26, srv1153760924=1, srv270100271=18, srv539228648=23, srv961186815=29, srv878894629=28, srv1898232070=16, srv278466873=19, srv661200055=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:56:52,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:56:52,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:56:52,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:56:52,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:56:52,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-05 11:56:52,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.100854670793867, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:56:52,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912510833038743, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.100854670793867, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-05 11:58:52,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6650427 different iterations.  Found a solution that moves 1846 regions; Going from a computed imbalance of 0.9912510833038743 to a new imbalance of 4.308656090753194E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6837037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:58:52,408 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1085298644:90 , srv1153760924:90 , srv1175380324:90 , srv1183703484:90 , srv1267018069:90 , srv127783693:90 , srv136269792:90 , srv1412949667:90 , srv1487656786:90 , srv1516392017:90 , srv1595369852:90 , srv1598980863:90 , srv1665594810:90 , srv174401704:90 , srv1795715519:90 , srv1817609365:90 , srv1898232070:90 , srv1989089832:90 , srv270100271:90 , srv278466873:90 , srv355761467:90 , srv485554897:90 , srv539185824:90 , srv539228648:90 , srv599885688:90 , srv661200055:90 , srv750362496:90 , srv824655252:90 , srv878894629:90 , srv961186815:90 }
2023-04-05 11:58:52,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-05 11:58:52,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv824655252=27, srv539185824=22, srv1183703484=3, srv1817609365=15, srv1795715519=14, srv1516392017=9, srv1412949667=7, srv136269792=6, srv599885688=24, srv1665594810=12, srv1595369852=10, srv1598980863=11, srv1175380324=2, srv1085298644=0, srv485554897=21, srv174401704=13, srv355761467=20, srv1267018069=4, srv127783693=5, srv1989089832=17, srv1487656786=8, srv750362496=26, srv1153760924=1, srv270100271=18, srv539228648=23, srv961186815=29, srv878894629=28, srv1898232070=16, srv278466873=19, srv661200055=25} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-05 11:58:52,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-05 11:58:52,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-05 11:58:52,425 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-05 11:58:52,425 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1085298644:90 , srv1153760924:90 , srv1175380324:90 , srv1183703484:90 , srv1267018069:90 , srv127783693:90 , srv136269792:90 , srv1412949667:90 , srv1487656786:90 , srv1516392017:90 , srv1595369852:90 , srv1598980863:90 , srv1665594810:90 , srv174401704:90 , srv1795715519:90 , srv1817609365:90 , srv1898232070:90 , srv1989089832:90 , srv270100271:90 , srv278466873:90 , srv355761467:90 , srv485554897:90 , srv539185824:90 , srv539228648:90 , srv599885688:90 , srv661200055:90 , srv750362496:90 , srv824655252:90 , srv878894629:90 , srv961186815:90 }
2023-04-05 11:58:52,438 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=279 (was 342), ProcessCount=169 (was 169), AvailableMemoryMB=1679 (was 1571) - AvailableMemoryMB LEAK? -