2024-01-10 08:53:24,141 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-10 08:53:24,486 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-10 08:53:24,607 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-10 08:53:24,686 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-10 08:53:24,693 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-10 08:53:24,708 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=306, ProcessCount=170, AvailableMemoryMB=8048
2024-01-10 08:53:24,710 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-10 08:53:24,711 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-10 08:53:24,851 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015186921:84 , srv1025401075:84 , srv1049905805:84 , srv1087794565:84 , srv1108886654:84 , srv1119054881:84 , srv1234509298:84 , srv1279921560:84 , srv1330709412:84 , srv1337161496:84 , srv1339366142:84 , srv1359818052:84 , srv1396740732:84 , srv1411496235:84 , srv1420686815:84 , srv1446185181:84 , srv1457820876:84 , srv1458214939:84 , srv1474180254:84 , srv1481918998:84 , srv1496272713:84 , srv150883381:84 , srv1574422309:84 , srv161335253:84 , srv1682192756:84 , srv1686184736:84 , srv1686244673:84 , srv1687540757:84 , srv1711161361:84 , srv1723819699:84 , srv1730822140:84 , srv1734713695:84 , srv1737039963:84 , srv1742124770:84 , srv1748683730:84 , srv1770457841:84 , srv1798725381:84 , srv1817342467:84 , srv1822624038:84 , srv1829703318:84 , srv1866648541:84 , srv1896768323:84 , srv1942071455:84 , srv1944795239:84 , srv1960965487:84 , srv1974324071:84 , srv200390566:84 , srv2011169176:84 , srv2035967774:84 , srv2050687081:84 , srv2088696186:84 , srv2120486778:84 , srv273486657:84 , srv281917050:84 , srv285507923:84 , srv296551470:84 , srv301042030:84 , srv311972799:84 , srv355933841:84 , srv36040822:84 , srv360649246:84 , srv381040022:84 , srv384213783:84 , srv389513780:84 , srv392066852:84 , srv412352068:84 , srv441090324:84 , srv446838602:84 , srv449714610:84 , srv46471441:84 , srv503104373:84 , srv507853889:84 , srv541738252:84 , srv591583007:84 , srv603021661:84 , srv605250491:84 , srv608316568:84 , srv613288353:84 , srv615866309:84 , srv670284195:84 , srv674761370:84 , srv693999009:84 , srv708721458:84 , srv718847717:84 , srv748560644:84 , srv77308448:84 , srv787779256:84 , srv820182562:84 , srv878546720:84 , srv907805683:84 , srv912749410:84 , srv934754938:84 , srv954507903:84 , srv976917096:84 , srv982433161:84 , srv983808753:84 , srv985647586:84 , srv989189741:84 , srv996176963:84 , srv1025912486:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-10 08:53:24,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:24,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:24,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:24,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:24,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:24,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:24,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:24,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:24,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:24,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:24,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:24,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:24,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:24,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:24,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:24,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:24,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:24,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:24,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:24,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:24,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:24,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:24,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:24,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:24,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:24,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:24,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:24,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:24,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:25,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:25,269 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-10 08:53:25,269 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-10 08:53:25,691 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 681 ms to try 5000 different iterations.  Found a solution that moves 4679 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18365826021118956. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405126766235322, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05979013052497077, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5198888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405126766235322, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:25,708 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv615866309:81 , srv1574422309:82 , srv1711161361:82 , srv296551470:82 , srv1049905805:83 , srv1457820876:83 , srv150883381:83 , srv1734713695:83 , srv1798725381:83 , srv2088696186:83 , srv441090324:83 , srv820182562:83 , srv1108886654:84 , srv1686184736:84 , srv1817342467:84 , srv389513780:84 , srv449714610:84 , srv718847717:84 , srv878546720:84 , srv934754938:84 , srv954507903:84 , srv982433161:84 , srv1234509298:85 , srv1339366142:85 , srv1481918998:85 , srv161335253:85 , srv1686244673:85 , srv1737039963:85 , srv1742124770:85 , srv1829703318:85 , srv1942071455:85 , srv1944795239:85 , srv2011169176:85 , srv2035967774:85 , srv2050687081:85 , srv273486657:85 , srv301042030:85 , srv311972799:85 , srv381040022:85 , srv670284195:85 , srv708721458:85 , srv1025401075:86 , srv1337161496:86 , srv1411496235:86 , srv1420686815:86 , srv1474180254:86 , srv1496272713:86 , srv1687540757:86 , srv1730822140:86 , srv1896768323:86 , srv200390566:86 , srv285507923:86 , srv360649246:86 , srv392066852:86 , srv503104373:86 , srv603021661:86 , srv674761370:86 , srv748560644:86 , srv77308448:86 , srv912749410:86 , srv1446185181:87 , srv1682192756:87 , srv1822624038:87 , srv1866648541:87 , srv1960965487:87 , srv1974324071:87 , srv355933841:87 , srv384213783:87 , srv46471441:87 , srv605250491:87 , srv608316568:87 , srv693999009:87 , srv985647586:87 , srv1396740732:88 , srv412352068:88 , srv591583007:88 , srv983808753:88 , srv996176963:88 , srv1015186921:89 , srv1330709412:89 , srv1458214939:89 , srv36040822:89 , srv446838602:89 , srv507853889:89 , srv613288353:89 , srv787779256:89 , srv907805683:89 , srv976917096:89 , srv1087794565:90 , srv1748683730:90 , srv281917050:90 , srv541738252:90 , srv1723819699:91 , srv2120486778:91 , srv1279921560:92 , srv1359818052:93 , srv1119054881:94 , srv1770457841:94 , srv989189741:94 , srv1025912486:450 }
2024-01-10 08:53:25,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:25,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:25,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:25,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:25,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:25,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:25,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:25,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:25,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:25,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:25,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:25,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:25,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:25,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:25,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:25,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:25,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:25,917 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405126766235322, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05979013052497077, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405126766235322, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:25,917 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-10 08:53:25,918 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18362549715293064, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0405126766235322, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05979013052497077, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0405126766235322, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-10 08:53:26,169 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 386 ms to try 5000 different iterations.  Found a solution that moves 2622 regions; Going from a computed imbalance of 0.18362549715293064 to a new imbalance of 0.016718943309047336. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02305239454754879); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035435403045557735, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29133333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02305239454754879); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:26,184 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv989189741:78 , srv1119054881:79 , srv1359818052:79 , srv1770457841:79 , srv1279921560:80 , srv1723819699:80 , srv2120486778:80 , srv541738252:80 , srv1748683730:81 , srv281917050:81 , srv787779256:81 , srv1015186921:82 , srv613288353:82 , srv976917096:82 , srv1087794565:83 , srv1330709412:83 , srv1396740732:83 , srv1974324071:83 , srv591583007:83 , srv1458214939:84 , srv1822624038:84 , srv1866648541:84 , srv36040822:84 , srv360649246:84 , srv384213783:84 , srv412352068:84 , srv446838602:84 , srv507853889:84 , srv605250491:84 , srv693999009:84 , srv907805683:84 , srv1234509298:85 , srv1481918998:85 , srv1496272713:85 , srv1960965487:85 , srv200390566:85 , srv285507923:85 , srv355933841:85 , srv381040022:85 , srv603021661:85 , srv77308448:85 , srv878546720:85 , srv996176963:85 , srv1337161496:86 , srv1339366142:86 , srv1474180254:86 , srv1682192756:86 , srv1686244673:86 , srv1687540757:86 , srv2050687081:86 , srv608316568:86 , srv708721458:86 , srv912749410:86 , srv954507903:86 , srv983808753:86 , srv985647586:86 , srv1411496235:87 , srv1574422309:87 , srv1730822140:87 , srv1742124770:87 , srv1817342467:87 , srv1829703318:87 , srv1896768323:87 , srv1942071455:87 , srv1944795239:87 , srv2011169176:87 , srv2035967774:87 , srv46471441:87 , srv503104373:87 , srv718847717:87 , srv748560644:87 , srv1025401075:88 , srv1420686815:88 , srv150883381:88 , srv161335253:88 , srv1686184736:88 , srv1734713695:88 , srv1737039963:88 , srv273486657:88 , srv296551470:88 , srv311972799:88 , srv392066852:88 , srv441090324:88 , srv670284195:88 , srv674761370:88 , srv820182562:88 , srv982433161:88 , srv1049905805:89 , srv1108886654:89 , srv1711161361:89 , srv2088696186:89 , srv301042030:89 , srv389513780:89 , srv449714610:89 , srv934754938:89 , srv1446185181:90 , srv1457820876:90 , srv1798725381:90 , srv615866309:91 , srv1025912486:529 }
2024-01-10 08:53:26,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:26,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:26,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:26,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:26,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:26,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:26,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:26,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:26,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:26,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:26,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:26,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:26,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:26,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:26,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:26,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:26,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:26,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02305239454754879); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035435403045557735, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02305239454754879); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:26,346 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-10 08:53:26,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016700583672638956, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02305239454754879); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035435403045557735, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02305239454754879); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-10 08:53:26,703 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 455 ms to try 5000 different iterations.  Found a solution that moves 975 regions; Going from a computed imbalance of 0.016700583672638956 to a new imbalance of 7.989500217018457E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007668789048930225); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011101005505463153); MoveCostFunction : (multiplier=7.0, imbalance=0.10833333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007668789048930225); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:26,709 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1446185181:81 , srv1420686815:83 , srv1770457841:83 , srv301042030:83 , srv389513780:83 , srv1087794565:84 , srv1108886654:84 , srv1119054881:84 , srv1279921560:84 , srv1411496235:84 , srv161335253:84 , srv1742124770:84 , srv2011169176:84 , srv273486657:84 , srv36040822:84 , srv392066852:84 , srv449714610:84 , srv46471441:84 , srv503104373:84 , srv608316568:84 , srv670284195:84 , srv748560644:84 , srv907805683:84 , srv934754938:84 , srv983808753:84 , srv996176963:84 , srv1025401075:85 , srv1049905805:85 , srv1337161496:85 , srv1457820876:85 , srv1458214939:85 , srv1682192756:85 , srv1686184736:85 , srv1686244673:85 , srv1687540757:85 , srv1730822140:85 , srv1737039963:85 , srv1798725381:85 , srv1829703318:85 , srv1896768323:85 , srv1944795239:85 , srv1960965487:85 , srv2088696186:85 , srv311972799:85 , srv355933841:85 , srv446838602:85 , srv507853889:85 , srv603021661:85 , srv615866309:85 , srv674761370:85 , srv982433161:85 , srv985647586:85 , srv1015186921:86 , srv1330709412:86 , srv1339366142:86 , srv1359818052:86 , srv1474180254:86 , srv1481918998:86 , srv1496272713:86 , srv150883381:86 , srv1723819699:86 , srv1734713695:86 , srv1748683730:86 , srv1817342467:86 , srv1822624038:86 , srv1866648541:86 , srv1942071455:86 , srv200390566:86 , srv2035967774:86 , srv2050687081:86 , srv2120486778:86 , srv281917050:86 , srv285507923:86 , srv384213783:86 , srv412352068:86 , srv441090324:86 , srv591583007:86 , srv605250491:86 , srv708721458:86 , srv718847717:86 , srv820182562:86 , srv912749410:86 , srv976917096:86 , srv989189741:86 , srv1396740732:87 , srv296551470:87 , srv381040022:87 , srv613288353:87 , srv693999009:87 , srv77308448:87 , srv787779256:87 , srv954507903:87 , srv1234509298:88 , srv1574422309:88 , srv1711161361:88 , srv1974324071:88 , srv360649246:88 , srv541738252:88 , srv878546720:90 , srv1025912486:547 }
2024-01-10 08:53:26,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:26,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:26,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:26,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:26,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:26,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:26,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:26,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:26,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:26,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:26,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:26,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:26,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:26,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:26,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:26,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:26,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:26,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007668789048930225); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011101005505463153); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007668789048930225); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:26,850 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-10 08:53:26,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.921229257833988E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007668789048930225); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011101005505463153); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007668789048930225); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-10 08:53:27,088 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 337 ms to try 5000 different iterations.  Found a solution that moves 305 regions; Going from a computed imbalance of 7.921229257833988E-4 to a new imbalance of 5.1932190847595245E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.03388888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:27,089 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1025401075:83 , srv1458214939:83 , srv1711161361:83 , srv613288353:83 , srv674761370:83 , srv1234509298:84 , srv1446185181:84 , srv1457820876:84 , srv1474180254:84 , srv150883381:84 , srv1682192756:84 , srv1730822140:84 , srv1737039963:84 , srv1798725381:84 , srv1896768323:84 , srv1942071455:84 , srv1974324071:84 , srv273486657:84 , srv301042030:84 , srv311972799:84 , srv360649246:84 , srv381040022:84 , srv412352068:84 , srv446838602:84 , srv46471441:84 , srv507853889:84 , srv541738252:84 , srv615866309:84 , srv693999009:84 , srv77308448:84 , srv878546720:84 , srv907805683:84 , srv912749410:84 , srv983808753:84 , srv985647586:84 , srv989189741:84 , srv1015186921:85 , srv1049905805:85 , srv1087794565:85 , srv1108886654:85 , srv1119054881:85 , srv1330709412:85 , srv1337161496:85 , srv1339366142:85 , srv1359818052:85 , srv1396740732:85 , srv1411496235:85 , srv1420686815:85 , srv1496272713:85 , srv1574422309:85 , srv161335253:85 , srv1686184736:85 , srv1687540757:85 , srv1723819699:85 , srv1734713695:85 , srv1748683730:85 , srv1817342467:85 , srv1822624038:85 , srv1829703318:85 , srv1866648541:85 , srv1944795239:85 , srv1960965487:85 , srv200390566:85 , srv2035967774:85 , srv2050687081:85 , srv2088696186:85 , srv2120486778:85 , srv281917050:85 , srv285507923:85 , srv296551470:85 , srv355933841:85 , srv36040822:85 , srv384213783:85 , srv392066852:85 , srv441090324:85 , srv449714610:85 , srv503104373:85 , srv591583007:85 , srv603021661:85 , srv605250491:85 , srv670284195:85 , srv708721458:85 , srv748560644:85 , srv787779256:85 , srv820182562:85 , srv934754938:85 , srv976917096:85 , srv982433161:85 , srv996176963:85 , srv1279921560:86 , srv1481918998:86 , srv1686244673:86 , srv1742124770:86 , srv1770457841:86 , srv2011169176:86 , srv389513780:86 , srv608316568:86 , srv718847717:86 , srv954507903:86 , srv1025912486:616 }
2024-01-10 08:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:27,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:27,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:27,220 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-10 08:53:27,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.979653520131184E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-10 08:53:27,451 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 330 ms to try 5000 different iterations.  Found a solution that moves 40 regions; Going from a computed imbalance of 4.979653520131184E-5 to a new imbalance of 7.667843486710382E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0044444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:27,452 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1330709412:83 , srv1359818052:83 , srv1396740732:83 , srv2035967774:83 , srv392066852:83 , srv608316568:83 , srv718847717:83 , srv878546720:83 , srv954507903:83 , srv1015186921:84 , srv1025401075:84 , srv1049905805:84 , srv1087794565:84 , srv1108886654:84 , srv1119054881:84 , srv1234509298:84 , srv1279921560:84 , srv1337161496:84 , srv1339366142:84 , srv1411496235:84 , srv1420686815:84 , srv1446185181:84 , srv1457820876:84 , srv1474180254:84 , srv1481918998:84 , srv1496272713:84 , srv1574422309:84 , srv161335253:84 , srv1682192756:84 , srv1686184736:84 , srv1686244673:84 , srv1687540757:84 , srv1711161361:84 , srv1723819699:84 , srv1730822140:84 , srv1734713695:84 , srv1737039963:84 , srv1742124770:84 , srv1748683730:84 , srv1770457841:84 , srv1798725381:84 , srv1817342467:84 , srv1822624038:84 , srv1829703318:84 , srv1866648541:84 , srv1896768323:84 , srv1942071455:84 , srv1944795239:84 , srv1960965487:84 , srv1974324071:84 , srv200390566:84 , srv2011169176:84 , srv2050687081:84 , srv2088696186:84 , srv2120486778:84 , srv281917050:84 , srv285507923:84 , srv296551470:84 , srv311972799:84 , srv355933841:84 , srv36040822:84 , srv360649246:84 , srv384213783:84 , srv389513780:84 , srv412352068:84 , srv441090324:84 , srv446838602:84 , srv449714610:84 , srv46471441:84 , srv503104373:84 , srv507853889:84 , srv541738252:84 , srv591583007:84 , srv605250491:84 , srv670284195:84 , srv674761370:84 , srv693999009:84 , srv708721458:84 , srv748560644:84 , srv77308448:84 , srv787779256:84 , srv820182562:84 , srv912749410:84 , srv934754938:84 , srv976917096:84 , srv982433161:84 , srv983808753:84 , srv985647586:84 , srv989189741:84 , srv996176963:84 , srv1458214939:85 , srv150883381:85 , srv273486657:85 , srv301042030:85 , srv381040022:85 , srv603021661:85 , srv613288353:85 , srv615866309:85 , srv907805683:85 , srv1025912486:684 }
2024-01-10 08:53:27,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:27,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv503104373=71, srv591583007=74, srv1686244673=27, srv1974324071=46, srv748560644=85, srv1723819699=30, srv608316568=77, srv36040822=60, srv718847717=84, srv989189741=98, srv1396740732=13, srv670284195=80, srv1481918998=20, srv446838602=68, srv976917096=94, srv301042030=57, srv2088696186=51, srv603021661=75, srv1458214939=18, srv693999009=82, srv996176963=99, srv954507903=93, srv1119054881=6, srv1574422309=23, srv1682192756=25, srv296551470=56, srv1359818052=12, srv2011169176=48, srv1822624038=39, srv1108886654=5, srv507853889=72, srv605250491=76, srv1737039963=33, srv392066852=65, srv934754938=92, srv1025401075=1, srv384213783=63, srv1330709412=9, srv1829703318=40, srv1474180254=19, srv1730822140=31, srv1711161361=29, srv1866648541=41, srv200390566=47, srv360649246=61, srv613288353=78, srv2050687081=50, srv907805683=90, srv1457820876=17, srv1942071455=43, srv1420686815=15, srv1234509298=7, srv615866309=79, srv983808753=96, srv1817342467=38, srv1896768323=42, srv820182562=88, srv982433161=95, srv77308448=86, srv389513780=64, srv1015186921=0, srv1960965487=45, srv878546720=89, srv985647586=97, srv441090324=67, srv1279921560=8, srv1944795239=44, srv912749410=91, srv1025912486=2, srv381040022=62, srv1339366142=11, srv150883381=22, srv311972799=58, srv449714610=69, srv1742124770=34, srv1496272713=21, srv273486657=53, srv1686184736=26, srv1049905805=3, srv2035967774=49, srv787779256=87, srv1687540757=28, srv281917050=54, srv412352068=66, srv285507923=55, srv1087794565=4, srv1770457841=36, srv1411496235=14, srv161335253=24, srv1446185181=16, srv674761370=81, srv355933841=59, srv1337161496=10, srv708721458=83, srv1748683730=35, srv541738252=73, srv1798725381=37, srv2120486778=52, srv1734713695=32, srv46471441=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:27,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:27,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:27,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:27,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:27,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-10 08:53:27,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-10 08:53:27,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-10 08:53:27,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-10 08:53:27,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-10 08:53:27,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-10 08:53:27,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:27,590 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1330709412:83 , srv1359818052:83 , srv1396740732:83 , srv2035967774:83 , srv392066852:83 , srv608316568:83 , srv718847717:83 , srv878546720:83 , srv954507903:83 , srv1015186921:84 , srv1025401075:84 , srv1049905805:84 , srv1087794565:84 , srv1108886654:84 , srv1119054881:84 , srv1234509298:84 , srv1279921560:84 , srv1337161496:84 , srv1339366142:84 , srv1411496235:84 , srv1420686815:84 , srv1446185181:84 , srv1457820876:84 , srv1474180254:84 , srv1481918998:84 , srv1496272713:84 , srv1574422309:84 , srv161335253:84 , srv1682192756:84 , srv1686184736:84 , srv1686244673:84 , srv1687540757:84 , srv1711161361:84 , srv1723819699:84 , srv1730822140:84 , srv1734713695:84 , srv1737039963:84 , srv1742124770:84 , srv1748683730:84 , srv1770457841:84 , srv1798725381:84 , srv1817342467:84 , srv1822624038:84 , srv1829703318:84 , srv1866648541:84 , srv1896768323:84 , srv1942071455:84 , srv1944795239:84 , srv1960965487:84 , srv1974324071:84 , srv200390566:84 , srv2011169176:84 , srv2050687081:84 , srv2088696186:84 , srv2120486778:84 , srv281917050:84 , srv285507923:84 , srv296551470:84 , srv311972799:84 , srv355933841:84 , srv36040822:84 , srv360649246:84 , srv384213783:84 , srv389513780:84 , srv412352068:84 , srv441090324:84 , srv446838602:84 , srv449714610:84 , srv46471441:84 , srv503104373:84 , srv507853889:84 , srv541738252:84 , srv591583007:84 , srv605250491:84 , srv670284195:84 , srv674761370:84 , srv693999009:84 , srv708721458:84 , srv748560644:84 , srv77308448:84 , srv787779256:84 , srv820182562:84 , srv912749410:84 , srv934754938:84 , srv976917096:84 , srv982433161:84 , srv983808753:84 , srv985647586:84 , srv989189741:84 , srv996176963:84 , srv1458214939:85 , srv150883381:85 , srv273486657:85 , srv301042030:85 , srv381040022:85 , srv603021661:85 , srv613288353:85 , srv615866309:85 , srv907805683:85 , srv1025912486:684 }
2024-01-10 08:53:27,626 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=306 (was 306), ProcessCount=170 (was 170), AvailableMemoryMB=7413 (was 8048)
2024-01-10 08:53:27,635 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=306, ProcessCount=170, AvailableMemoryMB=7413
2024-01-10 08:53:27,636 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-10 08:53:27,636 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-10 08:53:27,644 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1269882554:84 , srv1352890258:84 , srv1433053509:84 , srv1464080171:84 , srv1465433840:84 , srv1484536758:84 , srv1510206612:84 , srv1657284336:84 , srv1809255979:84 , srv1845481890:84 , srv189580177:84 , srv1896183247:84 , srv1919701782:84 , srv1940101785:84 , srv2059050003:84 , srv2074853698:84 , srv2079222503:84 , srv2095435396:84 , srv33404301:84 , srv442550181:84 , srv475447220:84 , srv566958082:84 , srv604699592:84 , srv646697382:84 , srv755101106:84 , srv756794824:84 , srv79130372:84 , srv893487360:84 , srv930885235:84 , srv1805033110:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-10 08:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2079222503=17, srv1809255979=9, srv755101106=25, srv1657284336=7, srv2074853698=16, srv1464080171=3, srv604699592=23, srv1433053509=2, srv1845481890=10, srv1465433840=4, srv756794824=26, srv1919701782=13, srv1896183247=12, srv646697382=24, srv189580177=11, srv1510206612=6, srv1352890258=1, srv1484536758=5, srv566958082=22, srv1269882554=0, srv1940101785=14, srv2095435396=18, srv893487360=28, srv475447220=21, srv1805033110=8, srv442550181=20, srv2059050003=15, srv79130372=27, srv33404301=19, srv930885235=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:53:27,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:53:27,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:53:27,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-10 08:53:27,675 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1693500501402716, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:53:27,676 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912726659742483, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1693500501402716, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-10 08:55:27,762 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120100 ms to try 5610644 different iterations.  Found a solution that moves 1863 regions; Going from a computed imbalance of 0.9912726659742483 to a new imbalance of 4.3483349388262195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.69, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:55:27,771 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1269882554:90 , srv1352890258:90 , srv1433053509:90 , srv1464080171:90 , srv1465433840:90 , srv1484536758:90 , srv1510206612:90 , srv1657284336:90 , srv1805033110:90 , srv1809255979:90 , srv1845481890:90 , srv189580177:90 , srv1896183247:90 , srv1919701782:90 , srv1940101785:90 , srv2059050003:90 , srv2074853698:90 , srv2079222503:90 , srv2095435396:90 , srv33404301:90 , srv442550181:90 , srv475447220:90 , srv566958082:90 , srv604699592:90 , srv646697382:90 , srv755101106:90 , srv756794824:90 , srv79130372:90 , srv893487360:90 , srv930885235:90 }
2024-01-10 08:55:27,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-10 08:55:27,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2079222503=17, srv1809255979=9, srv755101106=25, srv1657284336=7, srv2074853698=16, srv1464080171=3, srv604699592=23, srv1433053509=2, srv1845481890=10, srv1465433840=4, srv756794824=26, srv1919701782=13, srv1896183247=12, srv646697382=24, srv189580177=11, srv1510206612=6, srv1352890258=1, srv1484536758=5, srv566958082=22, srv1269882554=0, srv1940101785=14, srv2095435396=18, srv893487360=28, srv475447220=21, srv1805033110=8, srv442550181=20, srv2059050003=15, srv79130372=27, srv33404301=19, srv930885235=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-10 08:55:27,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-10 08:55:27,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-10 08:55:27,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-10 08:55:27,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-10 08:55:27,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-10 08:55:27,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-10 08:55:27,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-10 08:55:27,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-10 08:55:27,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-10 08:55:27,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-10 08:55:27,972 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1269882554:90 , srv1352890258:90 , srv1433053509:90 , srv1464080171:90 , srv1465433840:90 , srv1484536758:90 , srv1510206612:90 , srv1657284336:90 , srv1805033110:90 , srv1809255979:90 , srv1845481890:90 , srv189580177:90 , srv1896183247:90 , srv1919701782:90 , srv1940101785:90 , srv2059050003:90 , srv2074853698:90 , srv2079222503:90 , srv2095435396:90 , srv33404301:90 , srv442550181:90 , srv475447220:90 , srv566958082:90 , srv604699592:90 , srv646697382:90 , srv755101106:90 , srv756794824:90 , srv79130372:90 , srv893487360:90 , srv930885235:90 }
2024-01-10 08:55:28,170 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=242 (was 306), ProcessCount=168 (was 170), AvailableMemoryMB=6815 (was 7413)