2024-04-20 20:53:36,184 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-20 20:53:36,474 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-20 20:53:36,591 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-20 20:53:36,662 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-20 20:53:36,670 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-20 20:53:36,686 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=245, ProcessCount=169, AvailableMemoryMB=8628
2024-04-20 20:53:36,688 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-20 20:53:36,688 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-20 20:53:36,832 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010529526:84 , srv1023385401:84 , srv1029047467:84 , srv1058038211:84 , srv1086916488:84 , srv1124518805:84 , srv1126345035:84 , srv1170344956:84 , srv1182252598:84 , srv1189135274:84 , srv1189849872:84 , srv1201623085:84 , srv1208009566:84 , srv1277283843:84 , srv1299103605:84 , srv1304295487:84 , srv1335477928:84 , srv1354724570:84 , srv1363566481:84 , srv1365801855:84 , srv1380062352:84 , srv1385858366:84 , srv1404597709:84 , srv142770234:84 , srv1450779691:84 , srv1465618204:84 , srv1471454677:84 , srv1475338758:84 , srv1477694892:84 , srv1482851048:84 , srv1494941106:84 , srv1506090523:84 , srv1518388725:84 , srv1528387531:84 , srv1531329255:84 , srv1551048799:84 , srv1553107744:84 , srv1590048491:84 , srv1606330630:84 , srv1637404353:84 , srv1644107822:84 , srv1688943033:84 , srv1700587856:84 , srv1705171572:84 , srv1711175665:84 , srv1729826865:84 , srv1730621379:84 , srv1732261808:84 , srv1779866834:84 , srv1815666176:84 , srv1840445438:84 , srv1850238175:84 , srv1867318524:84 , srv18821800:84 , srv191357540:84 , srv1920892840:84 , srv1998287427:84 , srv2014204926:84 , srv2023993834:84 , srv2028738624:84 , srv2037529435:84 , srv2047964759:84 , srv2068739185:84 , srv2074973953:84 , srv2105626544:84 , srv2138350556:84 , srv2145761450:84 , srv248551390:84 , srv248612922:84 , srv259521736:84 , srv303780686:84 , srv306878346:84 , srv339737936:84 , srv417361558:84 , srv470814049:84 , srv511568137:84 , srv527098692:84 , srv536836966:84 , srv548863526:84 , srv557783293:84 , srv561332485:84 , srv56279967:84 , srv614590703:84 , srv614654229:84 , srv618885914:84 , srv656057989:84 , srv680027273:84 , srv689287055:84 , srv716710378:84 , srv75080350:84 , srv78057867:84 , srv791306414:84 , srv840573134:84 , srv851599327:84 , srv891034466:84 , srv897486347:84 , srv902500764:84 , srv931588783:84 , srv96312521:84 , srv356583034:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-20 20:53:36,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:36,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:36,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:36,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:36,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:36,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:36,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:36,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:36,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:36,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:36,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:36,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:36,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:36,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:36,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:36,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:36,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:36,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:36,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:37,225 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:37,226 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-20 20:53:37,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-20 20:53:37,615 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 639 ms to try 5000 different iterations.  Found a solution that moves 4705 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17418042900787487. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037924058245258366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053648061270790366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5227777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037924058245258366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16916666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:37,636 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1170344956:82 , srv536836966:82 , srv891034466:82 , srv1023385401:83 , srv1182252598:83 , srv1385858366:83 , srv1475338758:83 , srv1518388725:83 , srv1606330630:83 , srv511568137:83 , srv527098692:83 , srv56279967:83 , srv716710378:83 , srv1201623085:84 , srv142770234:84 , srv1477694892:84 , srv1494941106:84 , srv1779866834:84 , srv1815666176:84 , srv18821800:84 , srv2037529435:84 , srv470814049:84 , srv618885914:84 , srv1126345035:85 , srv1304295487:85 , srv1482851048:85 , srv1711175665:85 , srv306878346:85 , srv614590703:85 , srv680027273:85 , srv689287055:85 , srv1029047467:86 , srv1058038211:86 , srv1086916488:86 , srv1189849872:86 , srv1208009566:86 , srv1471454677:86 , srv1531329255:86 , srv1553107744:86 , srv1590048491:86 , srv1637404353:86 , srv1998287427:86 , srv2138350556:86 , srv2145761450:86 , srv339737936:86 , srv902500764:86 , srv931588783:86 , srv96312521:86 , srv1010529526:87 , srv1299103605:87 , srv1404597709:87 , srv1450779691:87 , srv1551048799:87 , srv1644107822:87 , srv1700587856:87 , srv1705171572:87 , srv1729826865:87 , srv1850238175:87 , srv191357540:87 , srv1920892840:87 , srv2028738624:87 , srv2068739185:87 , srv2105626544:87 , srv248612922:87 , srv259521736:87 , srv561332485:87 , srv614654229:87 , srv840573134:87 , srv1124518805:88 , srv1335477928:88 , srv1363566481:88 , srv1506090523:88 , srv1840445438:88 , srv1867318524:88 , srv248551390:88 , srv303780686:88 , srv75080350:88 , srv897486347:88 , srv1365801855:89 , srv1730621379:89 , srv2014204926:89 , srv548863526:89 , srv557783293:89 , srv78057867:89 , srv851599327:89 , srv1277283843:90 , srv1354724570:90 , srv1528387531:90 , srv2047964759:90 , srv2074973953:90 , srv791306414:90 , srv1189135274:91 , srv1688943033:91 , srv2023993834:91 , srv417361558:91 , srv656057989:91 , srv1380062352:92 , srv1465618204:92 , srv1732261808:93 , srv356583034:427 }
2024-04-20 20:53:37,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:37,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:37,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:37,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:37,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:37,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:37,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:37,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:37,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:37,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:37,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:37,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:37,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:37,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:37,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:37,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:37,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037924058245258366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053648061270790366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037924058245258366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16916666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:37,802 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-20 20:53:37,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1741474838937248, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037924058245258366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053648061270790366, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037924058245258366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16916666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.238, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-20 20:53:38,065 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 375 ms to try 5000 different iterations.  Found a solution that moves 2611 regions; Going from a computed imbalance of 0.1741474838937248 to a new imbalance of 0.015098087899030667. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02193359134365773); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03657012082798931, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2901111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02193359134365773); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:38,087 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1380062352:79 , srv1732261808:80 , srv656057989:80 , srv1189135274:81 , srv1465618204:81 , srv1688943033:81 , srv2047964759:81 , srv1528387531:82 , srv1730621379:82 , srv2023993834:82 , srv75080350:82 , srv851599327:82 , srv1277283843:83 , srv1335477928:83 , srv1363566481:83 , srv1506090523:83 , srv1644107822:83 , srv1729826865:83 , srv2074973953:83 , srv417361558:83 , srv548863526:83 , srv614654229:83 , srv78057867:83 , srv791306414:83 , srv897486347:83 , srv1086916488:84 , srv1299103605:84 , srv1551048799:84 , srv1705171572:84 , srv1840445438:84 , srv1850238175:84 , srv1920892840:84 , srv2068739185:84 , srv2105626544:84 , srv557783293:84 , srv561332485:84 , srv931588783:84 , srv1126345035:85 , srv1365801855:85 , srv1471454677:85 , srv1482851048:85 , srv1590048491:85 , srv191357540:85 , srv2014204926:85 , srv248551390:85 , srv259521736:85 , srv306878346:85 , srv339737936:85 , srv1010529526:86 , srv1058038211:86 , srv1124518805:86 , srv1189849872:86 , srv1304295487:86 , srv1354724570:86 , srv1404597709:86 , srv1477694892:86 , srv1553107744:86 , srv1700587856:86 , srv1711175665:86 , srv1867318524:86 , srv2138350556:86 , srv2145761450:86 , srv248612922:86 , srv614590703:86 , srv680027273:86 , srv689287055:86 , srv96312521:86 , srv1029047467:87 , srv142770234:87 , srv1450779691:87 , srv1494941106:87 , srv1531329255:87 , srv1606330630:87 , srv1637404353:87 , srv18821800:87 , srv2028738624:87 , srv303780686:87 , srv840573134:87 , srv902500764:87 , srv1208009566:88 , srv1779866834:88 , srv2037529435:88 , srv1023385401:89 , srv1182252598:89 , srv1201623085:89 , srv1475338758:89 , srv1518388725:89 , srv1815666176:89 , srv1998287427:89 , srv527098692:89 , srv56279967:89 , srv618885914:89 , srv716710378:89 , srv1170344956:90 , srv1385858366:90 , srv470814049:90 , srv511568137:90 , srv536836966:91 , srv891034466:91 , srv356583034:542 }
2024-04-20 20:53:38,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:38,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:38,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:38,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:38,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:38,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:38,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:38,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:38,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:38,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02193359134365773); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03657012082798931, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02193359134365773); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:38,243 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-20 20:53:38,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015079805286268549, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02193359134365773); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03657012082798931, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02193359134365773); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04616666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-20 20:53:38,483 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 347 ms to try 5000 different iterations.  Found a solution that moves 982 regions; Going from a computed imbalance of 0.015079805286268549 to a new imbalance of 6.76857549307329E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005213965342504877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00992678020345126); MoveCostFunction : (multiplier=7.0, imbalance=0.10911111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005213965342504877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:38,488 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1354724570:81 , srv1998287427:82 , srv1208009566:83 , srv1450779691:83 , srv303780686:83 , srv417361558:83 , srv470814049:83 , srv840573134:83 , srv1010529526:84 , srv1029047467:84 , srv1277283843:84 , srv1365801855:84 , srv1385858366:84 , srv1404597709:84 , srv1531329255:84 , srv1637404353:84 , srv1732261808:84 , srv2014204926:84 , srv2023993834:84 , srv2028738624:84 , srv248551390:84 , srv248612922:84 , srv511568137:84 , srv618885914:84 , srv891034466:84 , srv1023385401:85 , srv1058038211:85 , srv1170344956:85 , srv1189135274:85 , srv1201623085:85 , srv1465618204:85 , srv1475338758:85 , srv1518388725:85 , srv1528387531:85 , srv1688943033:85 , srv1700587856:85 , srv1779866834:85 , srv1867318524:85 , srv191357540:85 , srv259521736:85 , srv536836966:85 , srv548863526:85 , srv557783293:85 , srv78057867:85 , srv791306414:85 , srv902500764:85 , srv1124518805:86 , srv1182252598:86 , srv1189849872:86 , srv1304295487:86 , srv142770234:86 , srv1494941106:86 , srv1553107744:86 , srv1590048491:86 , srv1711175665:86 , srv1730621379:86 , srv1815666176:86 , srv1840445438:86 , srv18821800:86 , srv2037529435:86 , srv2068739185:86 , srv2074973953:86 , srv2105626544:86 , srv2138350556:86 , srv2145761450:86 , srv339737936:86 , srv527098692:86 , srv56279967:86 , srv689287055:86 , srv716710378:86 , srv897486347:86 , srv96312521:86 , srv1086916488:87 , srv1126345035:87 , srv1299103605:87 , srv1335477928:87 , srv1363566481:87 , srv1380062352:87 , srv1471454677:87 , srv1477694892:87 , srv1482851048:87 , srv1551048799:87 , srv1644107822:87 , srv1705171572:87 , srv1729826865:87 , srv1850238175:87 , srv2047964759:87 , srv561332485:87 , srv614590703:87 , srv656057989:87 , srv75080350:87 , srv851599327:87 , srv931588783:87 , srv1506090523:88 , srv1606330630:88 , srv1920892840:88 , srv306878346:88 , srv614654229:88 , srv680027273:88 , srv356583034:535 }
2024-04-20 20:53:38,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:38,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:38,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:38,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:38,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:38,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:38,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:38,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:38,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:38,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:38,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:38,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:38,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:38,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:38,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:38,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:38,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:38,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:38,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:38,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:38,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:38,692 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005213965342504877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00992678020345126); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005213965342504877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:38,692 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-20 20:53:38,695 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.699814383412623E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005213965342504877); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00992678020345126); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005213965342504877); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-20 20:53:39,115 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 569 ms to try 5000 different iterations.  Found a solution that moves 276 regions; Going from a computed imbalance of 6.699814383412623E-4 to a new imbalance of 2.8472950401758044E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.030666666666666665, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:39,116 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1124518805:82 , srv2074973953:82 , srv1465618204:83 , srv1506090523:83 , srv1815666176:83 , srv1867318524:83 , srv1920892840:83 , srv2138350556:83 , srv527098692:83 , srv561332485:83 , srv680027273:83 , srv902500764:83 , srv1023385401:84 , srv1182252598:84 , srv1189135274:84 , srv1189849872:84 , srv1201623085:84 , srv1299103605:84 , srv1335477928:84 , srv1363566481:84 , srv1365801855:84 , srv1380062352:84 , srv1385858366:84 , srv1551048799:84 , srv1553107744:84 , srv1590048491:84 , srv1606330630:84 , srv1700587856:84 , srv1705171572:84 , srv1729826865:84 , srv1840445438:84 , srv1850238175:84 , srv18821800:84 , srv2014204926:84 , srv2028738624:84 , srv2037529435:84 , srv2047964759:84 , srv2145761450:84 , srv306878346:84 , srv536836966:84 , srv56279967:84 , srv656057989:84 , srv716710378:84 , srv791306414:84 , srv840573134:84 , srv851599327:84 , srv96312521:84 , srv1010529526:85 , srv1029047467:85 , srv1058038211:85 , srv1086916488:85 , srv1126345035:85 , srv1170344956:85 , srv1277283843:85 , srv1304295487:85 , srv1354724570:85 , srv1404597709:85 , srv142770234:85 , srv1450779691:85 , srv1471454677:85 , srv1475338758:85 , srv1477694892:85 , srv1482851048:85 , srv1494941106:85 , srv1518388725:85 , srv1528387531:85 , srv1531329255:85 , srv1637404353:85 , srv1688943033:85 , srv1711175665:85 , srv191357540:85 , srv2023993834:85 , srv2068739185:85 , srv248551390:85 , srv248612922:85 , srv259521736:85 , srv303780686:85 , srv339737936:85 , srv470814049:85 , srv511568137:85 , srv548863526:85 , srv557783293:85 , srv614590703:85 , srv614654229:85 , srv618885914:85 , srv689287055:85 , srv75080350:85 , srv78057867:85 , srv891034466:85 , srv897486347:85 , srv931588783:85 , srv1208009566:86 , srv1644107822:86 , srv1730621379:86 , srv1732261808:86 , srv1779866834:86 , srv1998287427:86 , srv2105626544:86 , srv417361558:86 , srv356583034:638 }
2024-04-20 20:53:39,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:39,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:39,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:39,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:39,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:39,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:39,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:39,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:39,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:39,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:39,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:39,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:39,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:39,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:39,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:39,274 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-20 20:53:39,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6540357095613055E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.694119368783758E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.694119368783758E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-20 20:53:39,562 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 31 regions; Going from a computed imbalance of 2.6540357095613055E-5 to a new imbalance of 5.859559632984987E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0034444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:39,563 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1208009566:83 , srv1471454677:83 , srv1644107822:83 , srv1730621379:83 , srv1732261808:83 , srv1779866834:83 , srv1998287427:83 , srv2105626544:83 , srv303780686:83 , srv417361558:83 , srv557783293:83 , srv614590703:83 , srv614654229:83 , srv1010529526:84 , srv1029047467:84 , srv1058038211:84 , srv1086916488:84 , srv1124518805:84 , srv1126345035:84 , srv1170344956:84 , srv1182252598:84 , srv1189849872:84 , srv1201623085:84 , srv1277283843:84 , srv1299103605:84 , srv1304295487:84 , srv1335477928:84 , srv1354724570:84 , srv1363566481:84 , srv1365801855:84 , srv1380062352:84 , srv1404597709:84 , srv142770234:84 , srv1450779691:84 , srv1475338758:84 , srv1477694892:84 , srv1482851048:84 , srv1494941106:84 , srv1506090523:84 , srv1518388725:84 , srv1528387531:84 , srv1531329255:84 , srv1551048799:84 , srv1553107744:84 , srv1606330630:84 , srv1637404353:84 , srv1688943033:84 , srv1700587856:84 , srv1705171572:84 , srv1711175665:84 , srv1815666176:84 , srv1840445438:84 , srv1850238175:84 , srv1867318524:84 , srv191357540:84 , srv1920892840:84 , srv2014204926:84 , srv2023993834:84 , srv2028738624:84 , srv2037529435:84 , srv2047964759:84 , srv2068739185:84 , srv2145761450:84 , srv248551390:84 , srv248612922:84 , srv259521736:84 , srv306878346:84 , srv339737936:84 , srv470814049:84 , srv511568137:84 , srv536836966:84 , srv548863526:84 , srv56279967:84 , srv618885914:84 , srv656057989:84 , srv680027273:84 , srv689287055:84 , srv716710378:84 , srv75080350:84 , srv78057867:84 , srv791306414:84 , srv851599327:84 , srv891034466:84 , srv897486347:84 , srv931588783:84 , srv96312521:84 , srv1023385401:85 , srv1189135274:85 , srv1385858366:85 , srv1465618204:85 , srv1590048491:85 , srv1729826865:85 , srv18821800:85 , srv2074973953:85 , srv2138350556:85 , srv527098692:85 , srv561332485:85 , srv840573134:85 , srv902500764:85 , srv356583034:684 }
2024-04-20 20:53:39,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:39,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv716710378=89, srv1354724570=17, srv1705171572=43, srv1277283843=13, srv1779866834=48, srv1553107744=36, srv1086916488=4, srv680027273=87, srv1730621379=46, srv931588783=98, srv1482851048=29, srv1471454677=26, srv75080350=90, srv1182252598=8, srv2074973953=63, srv511568137=76, srv1170344956=7, srv689287055=88, srv1299103605=14, srv1637404353=39, srv1208009566=12, srv1124518805=5, srv2105626544=64, srv306878346=71, srv248612922=68, srv1365801855=19, srv78057867=91, srv791306414=92, srv1189849872=10, srv1363566481=18, srv417361558=74, srv1010529526=0, srv2014204926=57, srv1189135274=9, srv1551048799=35, srv356583034=73, srv614654229=84, srv1732261808=47, srv1815666176=49, srv1201623085=11, srv2023993834=58, srv1850238175=51, srv1920892840=55, srv2028738624=59, srv1494941106=30, srv527098692=77, srv1590048491=37, srv614590703=83, srv2068739185=62, srv891034466=95, srv1380062352=20, srv1867318524=52, srv561332485=81, srv248551390=67, srv840573134=93, srv1335477928=16, srv1729826865=45, srv1477694892=28, srv851599327=94, srv1450779691=24, srv1840445438=50, srv557783293=80, srv1385858366=21, srv142770234=23, srv2037529435=60, srv1304295487=15, srv339737936=72, srv548863526=79, srv2047964759=61, srv1126345035=6, srv1475338758=27, srv1606330630=38, srv2145761450=66, srv191357540=54, srv897486347=96, srv1688943033=41, srv1058038211=3, srv1998287427=56, srv1023385401=1, srv1711175665=44, srv902500764=97, srv1029047467=2, srv1644107822=40, srv56279967=82, srv1465618204=25, srv1518388725=32, srv656057989=86, srv259521736=69, srv1700587856=42, srv18821800=53, srv1404597709=22, srv1528387531=33, srv618885914=85, srv536836966=78, srv96312521=99, srv1506090523=31, srv2138350556=65, srv303780686=70, srv1531329255=34, srv470814049=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:39,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:39,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-20 20:53:39,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-20 20:53:39,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-20 20:53:39,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-20 20:53:39,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-20 20:53:39,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-20 20:53:39,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-20 20:53:39,740 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=5.642492993526674E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:39,740 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1208009566:83 , srv1471454677:83 , srv1644107822:83 , srv1730621379:83 , srv1732261808:83 , srv1779866834:83 , srv1998287427:83 , srv2105626544:83 , srv303780686:83 , srv417361558:83 , srv557783293:83 , srv614590703:83 , srv614654229:83 , srv1010529526:84 , srv1029047467:84 , srv1058038211:84 , srv1086916488:84 , srv1124518805:84 , srv1126345035:84 , srv1170344956:84 , srv1182252598:84 , srv1189849872:84 , srv1201623085:84 , srv1277283843:84 , srv1299103605:84 , srv1304295487:84 , srv1335477928:84 , srv1354724570:84 , srv1363566481:84 , srv1365801855:84 , srv1380062352:84 , srv1404597709:84 , srv142770234:84 , srv1450779691:84 , srv1475338758:84 , srv1477694892:84 , srv1482851048:84 , srv1494941106:84 , srv1506090523:84 , srv1518388725:84 , srv1528387531:84 , srv1531329255:84 , srv1551048799:84 , srv1553107744:84 , srv1606330630:84 , srv1637404353:84 , srv1688943033:84 , srv1700587856:84 , srv1705171572:84 , srv1711175665:84 , srv1815666176:84 , srv1840445438:84 , srv1850238175:84 , srv1867318524:84 , srv191357540:84 , srv1920892840:84 , srv2014204926:84 , srv2023993834:84 , srv2028738624:84 , srv2037529435:84 , srv2047964759:84 , srv2068739185:84 , srv2145761450:84 , srv248551390:84 , srv248612922:84 , srv259521736:84 , srv306878346:84 , srv339737936:84 , srv470814049:84 , srv511568137:84 , srv536836966:84 , srv548863526:84 , srv56279967:84 , srv618885914:84 , srv656057989:84 , srv680027273:84 , srv689287055:84 , srv716710378:84 , srv75080350:84 , srv78057867:84 , srv791306414:84 , srv851599327:84 , srv891034466:84 , srv897486347:84 , srv931588783:84 , srv96312521:84 , srv1023385401:85 , srv1189135274:85 , srv1385858366:85 , srv1465618204:85 , srv1590048491:85 , srv1729826865:85 , srv18821800:85 , srv2074973953:85 , srv2138350556:85 , srv527098692:85 , srv561332485:85 , srv840573134:85 , srv902500764:85 , srv356583034:684 }
2024-04-20 20:53:39,766 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 245) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=8101 (was 8628)
2024-04-20 20:53:39,776 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=169, AvailableMemoryMB=8100
2024-04-20 20:53:39,777 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-20 20:53:39,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-20 20:53:39,785 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1060802785:84 , srv1098324831:84 , srv1109314095:84 , srv1115157617:84 , srv1140132971:84 , srv1150799:84 , srv1310176713:84 , srv1436442890:84 , srv1522170884:84 , srv1552363872:84 , srv1603257813:84 , srv1652183504:84 , srv1697193999:84 , srv173118687:84 , srv1796136165:84 , srv1822987061:84 , srv1823096277:84 , srv1866329821:84 , srv1942834112:84 , srv2056430320:84 , srv2114002891:84 , srv2143433825:84 , srv348152728:84 , srv457334938:84 , srv470882841:84 , srv500969436:84 , srv719381645:84 , srv898530045:84 , srv911285485:84 , srv780402255:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-20 20:53:39,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:53:39,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1866329821=17, srv1109314095=2, srv911285485=29, srv1652183504=11, srv2056430320=19, srv1115157617=3, srv1796136165=14, srv1522170884=8, srv2143433825=21, srv719381645=26, srv1552363872=9, srv898530045=28, srv1098324831=1, srv173118687=13, srv1310176713=6, srv348152728=22, srv470882841=24, srv1150799=5, srv500969436=25, srv1822987061=15, srv457334938=23, srv1823096277=16, srv2114002891=20, srv1140132971=4, srv780402255=27, srv1603257813=10, srv1436442890=7, srv1942834112=18, srv1060802785=0, srv1697193999=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-20 20:53:39,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1061118133904777, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:53:39,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912527398123404, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1061118133904777, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-20 20:55:39,799 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6168612 different iterations.  Found a solution that moves 1847 regions; Going from a computed imbalance of 0.9912527398123404 to a new imbalance of 4.3109901406398434E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6840740740740741, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:55:39,808 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1060802785:90 , srv1098324831:90 , srv1109314095:90 , srv1115157617:90 , srv1140132971:90 , srv1150799:90 , srv1310176713:90 , srv1436442890:90 , srv1522170884:90 , srv1552363872:90 , srv1603257813:90 , srv1652183504:90 , srv1697193999:90 , srv173118687:90 , srv1796136165:90 , srv1822987061:90 , srv1823096277:90 , srv1866329821:90 , srv1942834112:90 , srv2056430320:90 , srv2114002891:90 , srv2143433825:90 , srv348152728:90 , srv457334938:90 , srv470882841:90 , srv500969436:90 , srv719381645:90 , srv780402255:90 , srv898530045:90 , srv911285485:90 }
2024-04-20 20:55:39,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-20 20:55:39,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1866329821=17, srv1109314095=2, srv911285485=29, srv1652183504=11, srv2056430320=19, srv1115157617=3, srv1796136165=14, srv1522170884=8, srv2143433825=21, srv719381645=26, srv1552363872=9, srv898530045=28, srv1098324831=1, srv173118687=13, srv1310176713=6, srv348152728=22, srv470882841=24, srv1150799=5, srv500969436=25, srv1822987061=15, srv457334938=23, srv1823096277=16, srv2114002891=20, srv1140132971=4, srv780402255=27, srv1603257813=10, srv1436442890=7, srv1942834112=18, srv1060802785=0, srv1697193999=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-20 20:55:39,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-20 20:55:39,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-20 20:55:39,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-20 20:55:39,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-20 20:55:39,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-20 20:55:39,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-20 20:55:39,835 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-20 20:55:39,835 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1060802785:90 , srv1098324831:90 , srv1109314095:90 , srv1115157617:90 , srv1140132971:90 , srv1150799:90 , srv1310176713:90 , srv1436442890:90 , srv1522170884:90 , srv1552363872:90 , srv1603257813:90 , srv1652183504:90 , srv1697193999:90 , srv173118687:90 , srv1796136165:90 , srv1822987061:90 , srv1823096277:90 , srv1866329821:90 , srv1942834112:90 , srv2056430320:90 , srv2114002891:90 , srv2143433825:90 , srv348152728:90 , srv457334938:90 , srv470882841:90 , srv500969436:90 , srv719381645:90 , srv780402255:90 , srv898530045:90 , srv911285485:90 }
2024-04-20 20:55:39,853 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=183 (was 250), ProcessCount=169 (was 169), AvailableMemoryMB=8015 (was 8100)