2023-09-16 07:53:12,198 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-16 07:53:12,522 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-16 07:53:12,640 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-16 07:53:12,700 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-16 07:53:12,709 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-16 07:53:12,724 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=338, ProcessCount=171, AvailableMemoryMB=10055
2023-09-16 07:53:12,726 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-16 07:53:12,726 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-16 07:53:12,889 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1039870619:84 , srv1041847400:84 , srv1070965005:84 , srv1085460876:84 , srv1112263862:84 , srv1112458417:84 , srv1113562160:84 , srv1121548007:84 , srv1138628325:84 , srv117881774:84 , srv1214780605:84 , srv1270454466:84 , srv1276477786:84 , srv1358957111:84 , srv1389301790:84 , srv1397638836:84 , srv1410656312:84 , srv1411691766:84 , srv1416822136:84 , srv1432457995:84 , srv1449773532:84 , srv1455553381:84 , srv1464981634:84 , srv1466055571:84 , srv1472919899:84 , srv148153797:84 , srv1520530002:84 , srv1533173155:84 , srv1537095723:84 , srv1559020867:84 , srv1574174801:84 , srv1601010434:84 , srv1630906419:84 , srv1653005043:84 , srv1724491382:84 , srv1730829101:84 , srv1731034681:84 , srv1761377029:84 , srv1767053248:84 , srv1778587273:84 , srv1786562279:84 , srv1789479904:84 , srv1811705298:84 , srv1827542160:84 , srv1849844902:84 , srv1865458320:84 , srv1905047588:84 , srv1928849391:84 , srv1932543739:84 , srv1939945810:84 , srv1947353173:84 , srv2017656070:84 , srv2041919863:84 , srv2050056174:84 , srv205542454:84 , srv2081757547:84 , srv2082815936:84 , srv2092870165:84 , srv2095467452:84 , srv2101080584:84 , srv2119072847:84 , srv212742585:84 , srv2135165979:84 , srv2141571812:84 , srv253625119:84 , srv264198314:84 , srv279557132:84 , srv293554322:84 , srv302494411:84 , srv32998963:84 , srv356825375:84 , srv394415472:84 , srv41446244:84 , srv437087165:84 , srv444429591:84 , srv498478364:84 , srv51284466:84 , srv525170142:84 , srv547385048:84 , srv560384849:84 , srv561059490:84 , srv567828129:84 , srv612090974:84 , srv614431312:84 , srv62618728:84 , srv667770360:84 , srv682638845:84 , srv692345764:84 , srv767919674:84 , srv788963682:84 , srv790361803:84 , srv827546197:84 , srv840600891:84 , srv842580933:84 , srv937199347:84 , srv95750693:84 , srv958660819:84 , srv960774343:84 , srv967729925:84 , srv1167241022:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-16 07:53:12,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:12,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:13,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:13,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:13,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:13,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:13,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:13,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:13,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:13,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:13,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:13,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:13,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:13,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:13,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:13,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:13,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:13,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:13,330 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:13,331 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-16 07:53:13,331 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-16 07:53:13,686 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 646 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18400887467648722. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038576749560896455, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05307597840573596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038576749560896455, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24216666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:13,703 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv51284466:82 , srv1630906419:83 , srv1827542160:83 , srv2081757547:83 , srv253625119:83 , srv560384849:83 , srv614431312:83 , srv967729925:83 , srv1214780605:84 , srv1397638836:84 , srv1472919899:84 , srv1559020867:84 , srv2017656070:84 , srv2041919863:84 , srv356825375:84 , srv498478364:84 , srv667770360:84 , srv790361803:84 , srv1039870619:85 , srv1112263862:85 , srv1276477786:85 , srv148153797:85 , srv1520530002:85 , srv1574174801:85 , srv1767053248:85 , srv1928849391:85 , srv1947353173:85 , srv2135165979:85 , srv264198314:85 , srv302494411:85 , srv32998963:85 , srv444429591:85 , srv561059490:85 , srv567828129:85 , srv612090974:85 , srv767919674:85 , srv1410656312:86 , srv1411691766:86 , srv1601010434:86 , srv1653005043:86 , srv1778587273:86 , srv2050056174:86 , srv394415472:86 , srv437087165:86 , srv525170142:86 , srv547385048:86 , srv692345764:86 , srv827546197:86 , srv842580933:86 , srv95750693:86 , srv958660819:86 , srv1070965005:87 , srv1113562160:87 , srv1138628325:87 , srv1432457995:87 , srv1455553381:87 , srv1464981634:87 , srv1533173155:87 , srv1724491382:87 , srv1730829101:87 , srv1786562279:87 , srv1865458320:87 , srv1932543739:87 , srv2092870165:87 , srv2119072847:87 , srv41446244:87 , srv1121548007:88 , srv1389301790:88 , srv1466055571:88 , srv1761377029:88 , srv1905047588:88 , srv1939945810:88 , srv205542454:88 , srv2082815936:88 , srv2095467452:88 , srv212742585:88 , srv279557132:88 , srv293554322:88 , srv682638845:88 , srv788963682:88 , srv937199347:88 , srv1085460876:89 , srv1112458417:89 , srv117881774:89 , srv1270454466:89 , srv1358957111:89 , srv1416822136:89 , srv1537095723:89 , srv1731034681:89 , srv1789479904:89 , srv2101080584:89 , srv960774343:89 , srv1849844902:90 , srv840600891:90 , srv1811705298:91 , srv62618728:91 , srv2141571812:92 , srv1041847400:93 , srv1449773532:93 , srv1167241022:433 }
2023-09-16 07:53:13,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:13,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:13,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:13,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:13,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:13,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:13,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:13,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:13,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:13,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:13,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:13,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:13,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:13,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:13,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:13,990 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038576749560896455, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05307597840573596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038576749560896455, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24216666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:13,991 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-16 07:53:13,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1839760696053304, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038576749560896455, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05307597840573596, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038576749560896455, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17966666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24216666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-16 07:53:14,373 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 604 ms to try 5000 different iterations.  Found a solution that moves 2670 regions; Going from a computed imbalance of 0.1839760696053304 to a new imbalance of 0.015488908565356622. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021502949913196734); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032094837021114424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2966666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021502949913196734); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:14,393 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1849844902:80 , srv1041847400:81 , srv1270454466:81 , srv1811705298:81 , srv2141571812:81 , srv117881774:82 , srv1416822136:82 , srv1761377029:82 , srv205542454:82 , srv2095467452:82 , srv2101080584:82 , srv62618728:82 , srv1112458417:83 , srv1113562160:83 , srv1121548007:83 , srv1449773532:83 , srv1731034681:83 , srv1789479904:83 , srv1939945810:83 , srv2082815936:83 , srv2119072847:83 , srv840600891:83 , srv960774343:83 , srv1070965005:84 , srv1085460876:84 , srv1389301790:84 , srv1533173155:84 , srv1537095723:84 , srv1865458320:84 , srv1932543739:84 , srv212742585:84 , srv437087165:84 , srv682638845:84 , srv937199347:84 , srv1358957111:85 , srv1455553381:85 , srv148153797:85 , srv1601010434:85 , srv1905047588:85 , srv1947353173:85 , srv2050056174:85 , srv32998963:85 , srv95750693:85 , srv1112263862:86 , srv1276477786:86 , srv1411691766:86 , srv1432457995:86 , srv1520530002:86 , srv1653005043:86 , srv1724491382:86 , srv1730829101:86 , srv1778587273:86 , srv1786562279:86 , srv2041919863:86 , srv279557132:86 , srv293554322:86 , srv394415472:86 , srv444429591:86 , srv498478364:86 , srv525170142:86 , srv547385048:86 , srv567828129:86 , srv692345764:86 , srv788963682:86 , srv1397638836:87 , srv1410656312:87 , srv1464981634:87 , srv1466055571:87 , srv1928849391:87 , srv2081757547:87 , srv2135165979:87 , srv264198314:87 , srv356825375:87 , srv41446244:87 , srv612090974:87 , srv790361803:87 , srv827546197:87 , srv958660819:87 , srv1039870619:88 , srv1472919899:88 , srv1574174801:88 , srv2017656070:88 , srv253625119:88 , srv302494411:88 , srv51284466:88 , srv561059490:88 , srv614431312:88 , srv767919674:88 , srv842580933:88 , srv967729925:88 , srv1138628325:89 , srv1214780605:89 , srv1630906419:89 , srv1827542160:89 , srv2092870165:89 , srv667770360:89 , srv1767053248:91 , srv560384849:91 , srv1559020867:92 , srv1167241022:532 }
2023-09-16 07:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:14,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:14,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:14,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:14,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:14,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:14,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:14,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:14,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:14,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:14,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:14,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:14,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:14,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:14,697 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021502949913196734); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032094837021114424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021502949913196734); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:14,697 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-16 07:53:14,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015470212825764567, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021502949913196734); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032094837021114424, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021502949913196734); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-16 07:53:15,016 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 541 ms to try 5000 different iterations.  Found a solution that moves 1007 regions; Going from a computed imbalance of 0.015470212825764567 to a new imbalance of 6.314381264031232E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007327855655802953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.11188888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007327855655802953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,019 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1449773532:81 , srv1767053248:81 , srv1138628325:82 , srv1466055571:82 , srv1559020867:82 , srv2092870165:83 , srv279557132:83 , srv293554322:83 , srv560384849:83 , srv788963682:83 , srv842580933:83 , srv1041847400:84 , srv1085460876:84 , srv1214780605:84 , srv1358957111:84 , srv1410656312:84 , srv1464981634:84 , srv1537095723:84 , srv1574174801:84 , srv1724491382:84 , srv1730829101:84 , srv2135165979:84 , srv302494411:84 , srv41446244:84 , srv561059490:84 , srv62618728:84 , srv767919674:84 , srv827546197:84 , srv840600891:84 , srv937199347:84 , srv1039870619:85 , srv1121548007:85 , srv1411691766:85 , srv1432457995:85 , srv1455553381:85 , srv1472919899:85 , srv1653005043:85 , srv1731034681:85 , srv1778587273:85 , srv1786562279:85 , srv1789479904:85 , srv1827542160:85 , srv1905047588:85 , srv2050056174:85 , srv212742585:85 , srv2141571812:85 , srv264198314:85 , srv356825375:85 , srv444429591:85 , srv525170142:85 , srv612090974:85 , srv667770360:85 , srv682638845:85 , srv958660819:85 , srv960774343:85 , srv1070965005:86 , srv1112458417:86 , srv117881774:86 , srv1276477786:86 , srv1389301790:86 , srv1520530002:86 , srv1601010434:86 , srv1630906419:86 , srv1811705298:86 , srv1865458320:86 , srv1928849391:86 , srv1939945810:86 , srv2017656070:86 , srv2082815936:86 , srv394415472:86 , srv547385048:86 , srv567828129:86 , srv614431312:86 , srv692345764:86 , srv790361803:86 , srv95750693:86 , srv967729925:86 , srv1112263862:87 , srv1397638836:87 , srv1416822136:87 , srv1533173155:87 , srv1849844902:87 , srv1932543739:87 , srv1947353173:87 , srv2041919863:87 , srv2081757547:87 , srv2095467452:87 , srv2101080584:87 , srv2119072847:87 , srv253625119:87 , srv51284466:87 , srv1113562160:88 , srv1270454466:88 , srv148153797:88 , srv1761377029:88 , srv205542454:88 , srv32998963:88 , srv437087165:88 , srv498478364:89 , srv1167241022:558 }
2023-09-16 07:53:15,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:15,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:15,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:15,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:15,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:15,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:15,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:15,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:15,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:15,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007327855655802953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007327855655802953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,186 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-16 07:53:15,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.243869616955581E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007327855655802953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00753592220347252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007327855655802953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-16 07:53:15,515 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 457 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 6.243869616955581E-4 to a new imbalance of 3.397927370954162E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,516 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1270454466:83 , srv2092870165:83 , srv498478364:83 , srv1041847400:84 , srv1112263862:84 , srv1112458417:84 , srv1113562160:84 , srv1138628325:84 , srv1358957111:84 , srv1389301790:84 , srv1397638836:84 , srv1432457995:84 , srv1464981634:84 , srv148153797:84 , srv1533173155:84 , srv1559020867:84 , srv1630906419:84 , srv1786562279:84 , srv1811705298:84 , srv1849844902:84 , srv1905047588:84 , srv1928849391:84 , srv1932543739:84 , srv2017656070:84 , srv205542454:84 , srv2101080584:84 , srv2119072847:84 , srv2141571812:84 , srv253625119:84 , srv302494411:84 , srv32998963:84 , srv394415472:84 , srv41446244:84 , srv437087165:84 , srv51284466:84 , srv547385048:84 , srv567828129:84 , srv667770360:84 , srv692345764:84 , srv958660819:84 , srv1039870619:85 , srv1085460876:85 , srv1121548007:85 , srv1214780605:85 , srv1276477786:85 , srv1410656312:85 , srv1411691766:85 , srv1416822136:85 , srv1449773532:85 , srv1455553381:85 , srv1466055571:85 , srv1472919899:85 , srv1520530002:85 , srv1537095723:85 , srv1574174801:85 , srv1601010434:85 , srv1653005043:85 , srv1724491382:85 , srv1730829101:85 , srv1731034681:85 , srv1761377029:85 , srv1767053248:85 , srv1778587273:85 , srv1827542160:85 , srv1865458320:85 , srv1939945810:85 , srv2041919863:85 , srv2081757547:85 , srv2082815936:85 , srv212742585:85 , srv2135165979:85 , srv264198314:85 , srv279557132:85 , srv293554322:85 , srv525170142:85 , srv560384849:85 , srv561059490:85 , srv612090974:85 , srv614431312:85 , srv62618728:85 , srv682638845:85 , srv767919674:85 , srv788963682:85 , srv790361803:85 , srv827546197:85 , srv840600891:85 , srv842580933:85 , srv95750693:85 , srv960774343:85 , srv967729925:85 , srv1070965005:86 , srv117881774:86 , srv1789479904:86 , srv1947353173:86 , srv2050056174:86 , srv2095467452:86 , srv356825375:86 , srv444429591:86 , srv937199347:86 , srv1167241022:619 }
2023-09-16 07:53:15,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:15,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:15,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:15,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:15,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:15,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:15,640 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,640 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-16 07:53:15,641 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.2347772838774294E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-16 07:53:15,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 319 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 3.2347772838774294E-5 to a new imbalance of 6.200130331567064E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,865 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1039870619:83 , srv1070965005:83 , srv117881774:83 , srv1416822136:83 , srv1761377029:83 , srv1789479904:83 , srv1947353173:83 , srv2095467452:83 , srv1041847400:84 , srv1085460876:84 , srv1112263862:84 , srv1112458417:84 , srv1113562160:84 , srv1138628325:84 , srv1214780605:84 , srv1276477786:84 , srv1358957111:84 , srv1389301790:84 , srv1397638836:84 , srv1410656312:84 , srv1411691766:84 , srv1432457995:84 , srv1449773532:84 , srv1455553381:84 , srv1464981634:84 , srv1466055571:84 , srv1472919899:84 , srv148153797:84 , srv1520530002:84 , srv1533173155:84 , srv1537095723:84 , srv1559020867:84 , srv1574174801:84 , srv1601010434:84 , srv1630906419:84 , srv1653005043:84 , srv1724491382:84 , srv1730829101:84 , srv1731034681:84 , srv1767053248:84 , srv1778587273:84 , srv1786562279:84 , srv1811705298:84 , srv1827542160:84 , srv1865458320:84 , srv1905047588:84 , srv1928849391:84 , srv1932543739:84 , srv1939945810:84 , srv2017656070:84 , srv2041919863:84 , srv2050056174:84 , srv205542454:84 , srv2081757547:84 , srv2082815936:84 , srv2092870165:84 , srv2101080584:84 , srv212742585:84 , srv2141571812:84 , srv253625119:84 , srv264198314:84 , srv279557132:84 , srv293554322:84 , srv32998963:84 , srv356825375:84 , srv394415472:84 , srv41446244:84 , srv437087165:84 , srv444429591:84 , srv498478364:84 , srv525170142:84 , srv547385048:84 , srv560384849:84 , srv561059490:84 , srv612090974:84 , srv614431312:84 , srv62618728:84 , srv667770360:84 , srv682638845:84 , srv692345764:84 , srv767919674:84 , srv788963682:84 , srv790361803:84 , srv827546197:84 , srv840600891:84 , srv842580933:84 , srv937199347:84 , srv95750693:84 , srv958660819:84 , srv960774343:84 , srv967729925:84 , srv1121548007:85 , srv1270454466:85 , srv1849844902:85 , srv2119072847:85 , srv2135165979:85 , srv302494411:85 , srv51284466:85 , srv567828129:85 , srv1167241022:684 }
2023-09-16 07:53:15,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:15,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1767053248=39, srv1085460876=3, srv1358957111=14, srv1827542160=44, srv1865458320=46, srv293554322=68, srv2041919863=53, srv302494411=69, srv41446244=73, srv1449773532=21, srv1432457995=20, srv567828129=82, srv148153797=26, srv62618728=85, srv1724491382=35, srv1778587273=40, srv960774343=98, srv1630906419=33, srv279557132=67, srv394415472=72, srv667770360=86, srv937199347=95, srv1112263862=4, srv2017656070=52, srv264198314=66, srv1397638836=16, srv444429591=75, srv51284466=77, srv95750693=96, srv1041847400=1, srv840600891=93, srv498478364=76, srv692345764=88, srv1789479904=42, srv1276477786=13, srv1466055571=24, srv356825375=71, srv1472919899=25, srv2082815936=57, srv1533173155=28, srv1113562160=6, srv1947353173=51, srv1601010434=32, srv1731034681=37, srv1928849391=48, srv2119072847=61, srv1574174801=31, srv682638845=87, srv1786562279=41, srv2095467452=59, srv2050056174=54, srv1905047588=47, srv2141571812=64, srv437087165=74, srv1389301790=15, srv1464981634=23, srv1039870619=0, srv1811705298=43, srv788963682=90, srv1932543739=49, srv967729925=99, srv1112458417=5, srv1121548007=7, srv1849844902=45, srv2135165979=63, srv525170142=78, srv612090974=83, srv827546197=92, srv1214780605=11, srv958660819=97, srv1730829101=36, srv2092870165=58, srv842580933=94, srv117881774=10, srv614431312=84, srv1270454466=12, srv2081757547=56, srv1520530002=27, srv2101080584=60, srv547385048=79, srv1537095723=29, srv561059490=81, srv1411691766=18, srv1761377029=38, srv205542454=55, srv767919674=89, srv1070965005=2, srv560384849=80, srv1410656312=17, srv1653005043=34, srv790361803=91, srv1455553381=22, srv1939945810=50, srv212742585=62, srv32998963=70, srv253625119=65, srv1416822136=19, srv1138628325=8, srv1559020867=30, srv1167241022=9} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:15,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-16 07:53:15,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-16 07:53:15,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-16 07:53:15,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-16 07:53:15,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-16 07:53:15,986 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:15,987 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1039870619:83 , srv1070965005:83 , srv117881774:83 , srv1416822136:83 , srv1761377029:83 , srv1789479904:83 , srv1947353173:83 , srv2095467452:83 , srv1041847400:84 , srv1085460876:84 , srv1112263862:84 , srv1112458417:84 , srv1113562160:84 , srv1138628325:84 , srv1214780605:84 , srv1276477786:84 , srv1358957111:84 , srv1389301790:84 , srv1397638836:84 , srv1410656312:84 , srv1411691766:84 , srv1432457995:84 , srv1449773532:84 , srv1455553381:84 , srv1464981634:84 , srv1466055571:84 , srv1472919899:84 , srv148153797:84 , srv1520530002:84 , srv1533173155:84 , srv1537095723:84 , srv1559020867:84 , srv1574174801:84 , srv1601010434:84 , srv1630906419:84 , srv1653005043:84 , srv1724491382:84 , srv1730829101:84 , srv1731034681:84 , srv1767053248:84 , srv1778587273:84 , srv1786562279:84 , srv1811705298:84 , srv1827542160:84 , srv1865458320:84 , srv1905047588:84 , srv1928849391:84 , srv1932543739:84 , srv1939945810:84 , srv2017656070:84 , srv2041919863:84 , srv2050056174:84 , srv205542454:84 , srv2081757547:84 , srv2082815936:84 , srv2092870165:84 , srv2101080584:84 , srv212742585:84 , srv2141571812:84 , srv253625119:84 , srv264198314:84 , srv279557132:84 , srv293554322:84 , srv32998963:84 , srv356825375:84 , srv394415472:84 , srv41446244:84 , srv437087165:84 , srv444429591:84 , srv498478364:84 , srv525170142:84 , srv547385048:84 , srv560384849:84 , srv561059490:84 , srv612090974:84 , srv614431312:84 , srv62618728:84 , srv667770360:84 , srv682638845:84 , srv692345764:84 , srv767919674:84 , srv788963682:84 , srv790361803:84 , srv827546197:84 , srv840600891:84 , srv842580933:84 , srv937199347:84 , srv95750693:84 , srv958660819:84 , srv960774343:84 , srv967729925:84 , srv1121548007:85 , srv1270454466:85 , srv1849844902:85 , srv2119072847:85 , srv2135165979:85 , srv302494411:85 , srv51284466:85 , srv567828129:85 , srv1167241022:684 }
2023-09-16 07:53:16,012 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=375 (was 338) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=9425 (was 10055)
2023-09-16 07:53:16,020 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=375, ProcessCount=171, AvailableMemoryMB=9424
2023-09-16 07:53:16,021 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-16 07:53:16,022 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-16 07:53:16,031 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1040406832:84 , srv1095857704:84 , srv1112387058:84 , srv1185955314:84 , srv120900643:84 , srv1279549042:84 , srv135776758:84 , srv1505756182:84 , srv1507609686:84 , srv1789812606:84 , srv1923778729:84 , srv1972082893:84 , srv241311388:84 , srv264515386:84 , srv317958506:84 , srv406927079:84 , srv41128440:84 , srv502694646:84 , srv551081357:84 , srv58533317:84 , srv590186803:84 , srv604875368:84 , srv678940275:84 , srv708562602:84 , srv715340255:84 , srv77267363:84 , srv777617746:84 , srv793209619:84 , srv838895603:84 , srv593799253:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-16 07:53:16,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:53:16,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv708562602=24, srv1923778729=10, srv1040406832=0, srv1112387058=2, srv120900643=4, srv1185955314=3, srv77267363=26, srv264515386=13, srv41128440=16, srv838895603=29, srv593799253=21, srv1507609686=8, srv604875368=22, srv678940275=23, srv135776758=6, srv317958506=14, srv502694646=17, srv777617746=27, srv1789812606=9, srv241311388=12, srv590186803=20, srv551081357=18, srv1505756182=7, srv58533317=19, srv1279549042=5, srv1095857704=1, srv715340255=25, srv1972082893=11, srv793209619=28, srv406927079=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-16 07:53:16,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1179268710540526, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:53:16,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912564626984305, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1179268710540526, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-16 07:55:16,045 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6152758 different iterations.  Found a solution that moves 1863 regions; Going from a computed imbalance of 0.9912564626984305 to a new imbalance of 4.3483349388262195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.69, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:55:16,051 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1040406832:90 , srv1095857704:90 , srv1112387058:90 , srv1185955314:90 , srv120900643:90 , srv1279549042:90 , srv135776758:90 , srv1505756182:90 , srv1507609686:90 , srv1789812606:90 , srv1923778729:90 , srv1972082893:90 , srv241311388:90 , srv264515386:90 , srv317958506:90 , srv406927079:90 , srv41128440:90 , srv502694646:90 , srv551081357:90 , srv58533317:90 , srv590186803:90 , srv593799253:90 , srv604875368:90 , srv678940275:90 , srv708562602:90 , srv715340255:90 , srv77267363:90 , srv777617746:90 , srv793209619:90 , srv838895603:90 }
2023-09-16 07:55:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-16 07:55:16,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv708562602=24, srv1923778729=10, srv1040406832=0, srv1112387058=2, srv120900643=4, srv1185955314=3, srv77267363=26, srv264515386=13, srv41128440=16, srv838895603=29, srv593799253=21, srv1507609686=8, srv604875368=22, srv678940275=23, srv135776758=6, srv317958506=14, srv502694646=17, srv777617746=27, srv1789812606=9, srv241311388=12, srv590186803=20, srv551081357=18, srv1505756182=7, srv58533317=19, srv1279549042=5, srv1095857704=1, srv715340255=25, srv1972082893=11, srv793209619=28, srv406927079=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-16 07:55:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-16 07:55:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-16 07:55:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-16 07:55:16,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-16 07:55:16,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-16 07:55:16,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-16 07:55:16,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-16 07:55:16,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-16 07:55:16,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-16 07:55:16,068 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-16 07:55:16,068 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1040406832:90 , srv1095857704:90 , srv1112387058:90 , srv1185955314:90 , srv120900643:90 , srv1279549042:90 , srv135776758:90 , srv1505756182:90 , srv1507609686:90 , srv1789812606:90 , srv1923778729:90 , srv1972082893:90 , srv241311388:90 , srv264515386:90 , srv317958506:90 , srv406927079:90 , srv41128440:90 , srv502694646:90 , srv551081357:90 , srv58533317:90 , srv590186803:90 , srv593799253:90 , srv604875368:90 , srv678940275:90 , srv708562602:90 , srv715340255:90 , srv77267363:90 , srv777617746:90 , srv793209619:90 , srv838895603:90 }
2023-09-16 07:55:16,081 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=149 (was 375), ProcessCount=171 (was 171), AvailableMemoryMB=8614 (was 9424)