2023-09-02 16:53:09,262 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-02 16:53:09,576 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-02 16:53:09,701 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-02 16:53:09,770 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-02 16:53:09,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-02 16:53:09,795 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=278, ProcessCount=173, AvailableMemoryMB=10221
2023-09-02 16:53:09,798 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-02 16:53:09,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-02 16:53:09,936 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1025396175:84 , srv1040062295:84 , srv1082439299:84 , srv1168670711:84 , srv1233961599:84 , srv1243037986:84 , srv1261858336:84 , srv1308665848:84 , srv1322310512:84 , srv1325856009:84 , srv1333359074:84 , srv1345168571:84 , srv1354635873:84 , srv1366129112:84 , srv1422129196:84 , srv1433184464:84 , srv1455020546:84 , srv1475339532:84 , srv1483325025:84 , srv1493094350:84 , srv1498043605:84 , srv1517716592:84 , srv1521540967:84 , srv1542775964:84 , srv1545929142:84 , srv1574099083:84 , srv1579378104:84 , srv1619473217:84 , srv163313346:84 , srv1635477986:84 , srv1650135965:84 , srv1653680871:84 , srv1658177069:84 , srv1658904352:84 , srv1667460962:84 , srv170419557:84 , srv1711666545:84 , srv1725636586:84 , srv1738709781:84 , srv1746189849:84 , srv1761719017:84 , srv1807623869:84 , srv1849361639:84 , srv1864549855:84 , srv1878988154:84 , srv1890659170:84 , srv1907360494:84 , srv1917649018:84 , srv193498913:84 , srv1941119707:84 , srv1947686916:84 , srv1956568182:84 , srv1961060608:84 , srv1992542366:84 , srv1998247394:84 , srv2054036935:84 , srv2058889191:84 , srv2064051005:84 , srv2092292682:84 , srv2111853692:84 , srv2113502721:84 , srv2124837565:84 , srv2128687845:84 , srv233997827:84 , srv269695012:84 , srv270264953:84 , srv303273802:84 , srv329629260:84 , srv353287067:84 , srv381383328:84 , srv402918819:84 , srv416102268:84 , srv418271253:84 , srv439259402:84 , srv473461290:84 , srv485892899:84 , srv518853731:84 , srv523767842:84 , srv536556399:84 , srv602548272:84 , srv627751161:84 , srv636512982:84 , srv639088198:84 , srv651455118:84 , srv658979175:84 , srv695595839:84 , srv718997813:84 , srv720737640:84 , srv765635822:84 , srv782423500:84 , srv835601345:84 , srv853299198:84 , srv858338485:84 , srv896466484:84 , srv919463122:84 , srv927274603:84 , srv954893921:84 , srv990213595:84 , srv997810221:84 , srv1615391159:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-02 16:53:09,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:09,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:10,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:10,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:10,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:10,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:10,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:10,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:10,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:10,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:10,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:10,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:10,256 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:10,256 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-02 16:53:10,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-02 16:53:10,787 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 706 ms to try 5000 different iterations.  Found a solution that moves 4681 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18107402980970952. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04045384136219606, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06075285253310408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5201111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04045384136219606, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:10,804 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1040062295:83 , srv329629260:83 , srv636512982:83 , srv1483325025:84 , srv1574099083:84 , srv1579378104:84 , srv1658904352:84 , srv1711666545:84 , srv1746189849:84 , srv1890659170:84 , srv1992542366:84 , srv439259402:84 , srv658979175:84 , srv782423500:84 , srv835601345:84 , srv1082439299:85 , srv1322310512:85 , srv1325856009:85 , srv1433184464:85 , srv1521540967:85 , srv1635477986:85 , srv1658177069:85 , srv1667460962:85 , srv1725636586:85 , srv1878988154:85 , srv2064051005:85 , srv2092292682:85 , srv270264953:85 , srv473461290:85 , srv765635822:85 , srv858338485:85 , srv919463122:85 , srv997810221:85 , srv1233961599:86 , srv1366129112:86 , srv1493094350:86 , srv1498043605:86 , srv1849361639:86 , srv1864549855:86 , srv193498913:86 , srv1941119707:86 , srv1961060608:86 , srv2058889191:86 , srv2111853692:86 , srv353287067:86 , srv381383328:86 , srv416102268:86 , srv518853731:86 , srv523767842:86 , srv627751161:86 , srv639088198:86 , srv651455118:86 , srv853299198:86 , srv927274603:86 , srv1168670711:87 , srv1308665848:87 , srv1333359074:87 , srv1345168571:87 , srv1354635873:87 , srv1422129196:87 , srv1517716592:87 , srv1545929142:87 , srv1761719017:87 , srv1907360494:87 , srv1917649018:87 , srv1998247394:87 , srv2113502721:87 , srv2128687845:87 , srv233997827:87 , srv269695012:87 , srv485892899:87 , srv602548272:87 , srv718997813:87 , srv896466484:87 , srv954893921:87 , srv1025396175:88 , srv1261858336:88 , srv1455020546:88 , srv1475339532:88 , srv1542775964:88 , srv163313346:88 , srv170419557:88 , srv1738709781:88 , srv1807623869:88 , srv1947686916:88 , srv1956568182:88 , srv2124837565:88 , srv303273802:88 , srv402918819:88 , srv536556399:88 , srv695595839:88 , srv1243037986:89 , srv1619473217:89 , srv720737640:89 , srv990213595:89 , srv1653680871:91 , srv418271253:91 , srv1650135965:92 , srv2054036935:92 , srv1615391159:450 }
2023-09-02 16:53:10,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:10,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:10,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:10,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:10,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:10,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:10,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:10,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:10,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:10,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:10,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:10,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:10,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:10,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:10,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:10,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:10,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:10,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:10,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:11,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04045384136219606, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06075285253310408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04045384136219606, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:11,009 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-02 16:53:11,009 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18104125274715133, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04045384136219606, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06075285253310408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04045384136219606, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-02 16:53:11,396 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 525 ms to try 5000 different iterations.  Found a solution that moves 2609 regions; Going from a computed imbalance of 0.18104125274715133 to a new imbalance of 0.01635555895821782. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041738322224156806, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28988888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:11,407 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1650135965:80 , srv1653680871:80 , srv720737640:80 , srv1542775964:81 , srv2054036935:81 , srv1619473217:82 , srv1761719017:82 , srv2124837565:82 , srv1308665848:83 , srv1956568182:83 , srv2128687845:83 , srv402918819:83 , srv418271253:83 , srv485892899:83 , srv896466484:83 , srv1025396175:84 , srv1082439299:84 , srv1345168571:84 , srv163313346:84 , srv170419557:84 , srv1725636586:84 , srv1807623869:84 , srv1941119707:84 , srv1947686916:84 , srv303273802:84 , srv416102268:84 , srv627751161:84 , srv718997813:84 , srv954893921:84 , srv1168670711:85 , srv1233961599:85 , srv1243037986:85 , srv1261858336:85 , srv1422129196:85 , srv1475339532:85 , srv1907360494:85 , srv193498913:85 , srv2058889191:85 , srv269695012:85 , srv381383328:85 , srv473461290:85 , srv523767842:85 , srv602548272:85 , srv639088198:85 , srv695595839:85 , srv858338485:85 , srv927274603:85 , srv990213595:85 , srv1354635873:86 , srv1366129112:86 , srv1433184464:86 , srv1455020546:86 , srv1483325025:86 , srv1498043605:86 , srv1738709781:86 , srv1849361639:86 , srv1864549855:86 , srv2111853692:86 , srv2113502721:86 , srv233997827:86 , srv353287067:86 , srv518853731:86 , srv651455118:86 , srv853299198:86 , srv919463122:86 , srv1322310512:87 , srv1325856009:87 , srv1333359074:87 , srv1517716592:87 , srv1521540967:87 , srv1635477986:87 , srv1658177069:87 , srv1667460962:87 , srv1711666545:87 , srv1878988154:87 , srv1917649018:87 , srv1961060608:87 , srv1998247394:87 , srv2064051005:87 , srv2092292682:87 , srv536556399:87 , srv658979175:87 , srv1658904352:88 , srv1992542366:88 , srv329629260:88 , srv439259402:88 , srv765635822:88 , srv997810221:88 , srv1040062295:89 , srv1545929142:89 , srv1574099083:89 , srv1579378104:89 , srv835601345:89 , srv1493094350:90 , srv1746189849:90 , srv1890659170:90 , srv270264953:90 , srv636512982:90 , srv782423500:90 , srv1615391159:526 }
2023-09-02 16:53:11,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:11,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:11,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:11,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:11,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:11,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:11,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:11,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:11,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:11,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:11,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:11,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:11,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:11,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:11,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:11,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:11,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:11,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:11,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:11,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:11,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:11,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:11,604 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041738322224156806, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:11,604 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-02 16:53:11,605 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01633729034975502, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041738322224156806, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-02 16:53:11,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 1005 regions; Going from a computed imbalance of 0.01633729034975502 to a new imbalance of 6.681695241104194E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012940336192913774); MoveCostFunction : (multiplier=7.0, imbalance=0.11166666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:11,903 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1493094350:80 , srv1545929142:82 , srv536556399:82 , srv1243037986:83 , srv1746189849:83 , srv1890659170:83 , srv1917649018:83 , srv1998247394:83 , srv270264953:83 , srv636512982:83 , srv782423500:83 , srv990213595:83 , srv1322310512:84 , srv1354635873:84 , srv1455020546:84 , srv1498043605:84 , srv1579378104:84 , srv1864549855:84 , srv2054036935:84 , srv2092292682:84 , srv2113502721:84 , srv233997827:84 , srv269695012:84 , srv418271253:84 , srv602548272:84 , srv835601345:84 , srv1025396175:85 , srv1040062295:85 , srv1168670711:85 , srv1261858336:85 , srv1333359074:85 , srv1345168571:85 , srv1422129196:85 , srv1475339532:85 , srv1517716592:85 , srv1521540967:85 , srv1574099083:85 , srv170419557:85 , srv1738709781:85 , srv1807623869:85 , srv1849361639:85 , srv1907360494:85 , srv1947686916:85 , srv1992542366:85 , srv2111853692:85 , srv329629260:85 , srv651455118:85 , srv695595839:85 , srv765635822:85 , srv853299198:85 , srv997810221:85 , srv1366129112:86 , srv1433184464:86 , srv1619473217:86 , srv1635477986:86 , srv1650135965:86 , srv1658177069:86 , srv1658904352:86 , srv1667460962:86 , srv1711666545:86 , srv1878988154:86 , srv193498913:86 , srv1961060608:86 , srv2058889191:86 , srv2064051005:86 , srv303273802:86 , srv353287067:86 , srv518853731:86 , srv523767842:86 , srv718997813:86 , srv919463122:86 , srv927274603:86 , srv1233961599:87 , srv1308665848:87 , srv1483325025:87 , srv163313346:87 , srv1653680871:87 , srv1956568182:87 , srv2124837565:87 , srv381383328:87 , srv402918819:87 , srv416102268:87 , srv439259402:87 , srv473461290:87 , srv627751161:87 , srv639088198:87 , srv658979175:87 , srv896466484:87 , srv954893921:87 , srv1082439299:88 , srv1325856009:88 , srv1725636586:88 , srv1761719017:88 , srv1941119707:88 , srv2128687845:88 , srv485892899:88 , srv720737640:88 , srv858338485:88 , srv1542775964:89 , srv1615391159:542 }
2023-09-02 16:53:11,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:11,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:11,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:11,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:11,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:11,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:11,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:11,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:11,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:11,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:11,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:12,036 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012940336192913774); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:12,037 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-02 16:53:12,037 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.61132363702174E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006790837632067591); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012940336192913774); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006790837632067591); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-02 16:53:12,514 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 578 ms to try 5000 different iterations.  Found a solution that moves 295 regions; Going from a computed imbalance of 6.61132363702174E-4 to a new imbalance of 5.769299447562873E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.03277777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:12,515 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1325856009:82 , srv1517716592:83 , srv163313346:83 , srv1738709781:83 , srv1961060608:83 , srv439259402:83 , srv1233961599:84 , srv1261858336:84 , srv1308665848:84 , srv1333359074:84 , srv1366129112:84 , srv1422129196:84 , srv1455020546:84 , srv1475339532:84 , srv1542775964:84 , srv1545929142:84 , srv1574099083:84 , srv1635477986:84 , srv1650135965:84 , srv1653680871:84 , srv1658177069:84 , srv1658904352:84 , srv1667460962:84 , srv1878988154:84 , srv1941119707:84 , srv1956568182:84 , srv2064051005:84 , srv2124837565:84 , srv2128687845:84 , srv270264953:84 , srv303273802:84 , srv353287067:84 , srv381383328:84 , srv402918819:84 , srv418271253:84 , srv485892899:84 , srv518853731:84 , srv639088198:84 , srv658979175:84 , srv695595839:84 , srv765635822:84 , srv858338485:84 , srv954893921:84 , srv997810221:84 , srv1025396175:85 , srv1040062295:85 , srv1082439299:85 , srv1168670711:85 , srv1243037986:85 , srv1354635873:85 , srv1433184464:85 , srv1483325025:85 , srv1521540967:85 , srv1579378104:85 , srv1619473217:85 , srv170419557:85 , srv1711666545:85 , srv1725636586:85 , srv1761719017:85 , srv1807623869:85 , srv1849361639:85 , srv1890659170:85 , srv1907360494:85 , srv1917649018:85 , srv193498913:85 , srv1947686916:85 , srv1992542366:85 , srv1998247394:85 , srv2054036935:85 , srv2058889191:85 , srv2111853692:85 , srv2113502721:85 , srv233997827:85 , srv416102268:85 , srv473461290:85 , srv523767842:85 , srv536556399:85 , srv651455118:85 , srv718997813:85 , srv720737640:85 , srv782423500:85 , srv835601345:85 , srv853299198:85 , srv896466484:85 , srv919463122:85 , srv927274603:85 , srv990213595:85 , srv1322310512:86 , srv1345168571:86 , srv1493094350:86 , srv1498043605:86 , srv1746189849:86 , srv1864549855:86 , srv2092292682:86 , srv269695012:86 , srv329629260:86 , srv602548272:86 , srv627751161:86 , srv636512982:86 , srv1615391159:624 }
2023-09-02 16:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:12,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:12,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:12,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:12,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:12,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:12,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:12,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:12,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:12,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:12,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:12,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:12,674 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:12,674 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-02 16:53:12,674 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.5627360325944776E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-02 16:53:12,945 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 400 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 5.5627360325944776E-5 to a new imbalance of 8.677692705885125E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:12,946 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1333359074:83 , srv1746189849:83 , srv627751161:83 , srv1025396175:84 , srv1040062295:84 , srv1082439299:84 , srv1168670711:84 , srv1233961599:84 , srv1243037986:84 , srv1261858336:84 , srv1322310512:84 , srv1325856009:84 , srv1345168571:84 , srv1354635873:84 , srv1366129112:84 , srv1433184464:84 , srv1455020546:84 , srv1475339532:84 , srv1483325025:84 , srv1493094350:84 , srv1498043605:84 , srv1517716592:84 , srv1521540967:84 , srv1542775964:84 , srv1545929142:84 , srv1574099083:84 , srv1579378104:84 , srv1619473217:84 , srv163313346:84 , srv1635477986:84 , srv1650135965:84 , srv1653680871:84 , srv1658177069:84 , srv1658904352:84 , srv1667460962:84 , srv170419557:84 , srv1711666545:84 , srv1725636586:84 , srv1738709781:84 , srv1761719017:84 , srv1807623869:84 , srv1849361639:84 , srv1864549855:84 , srv1878988154:84 , srv1890659170:84 , srv1907360494:84 , srv1917649018:84 , srv193498913:84 , srv1941119707:84 , srv1947686916:84 , srv1956568182:84 , srv1961060608:84 , srv1992542366:84 , srv1998247394:84 , srv2054036935:84 , srv2058889191:84 , srv2064051005:84 , srv2092292682:84 , srv2111853692:84 , srv2113502721:84 , srv2128687845:84 , srv233997827:84 , srv269695012:84 , srv270264953:84 , srv303273802:84 , srv329629260:84 , srv353287067:84 , srv381383328:84 , srv402918819:84 , srv416102268:84 , srv418271253:84 , srv439259402:84 , srv473461290:84 , srv485892899:84 , srv518853731:84 , srv523767842:84 , srv536556399:84 , srv602548272:84 , srv636512982:84 , srv639088198:84 , srv651455118:84 , srv658979175:84 , srv695595839:84 , srv718997813:84 , srv720737640:84 , srv765635822:84 , srv782423500:84 , srv835601345:84 , srv853299198:84 , srv858338485:84 , srv896466484:84 , srv919463122:84 , srv927274603:84 , srv954893921:84 , srv990213595:84 , srv997810221:84 , srv1308665848:85 , srv1422129196:85 , srv2124837565:85 , srv1615391159:684 }
2023-09-02 16:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1998247394=55, srv473461290=75, srv418271253=73, srv1738709781=39, srv1941119707=50, srv1322310512=8, srv193498913=49, srv1521540967=22, srv919463122=95, srv1040062295=1, srv1650135965=31, srv329629260=68, srv1917649018=48, srv1455020546=16, srv1261858336=6, srv2064051005=58, srv1517716592=21, srv651455118=84, srv1354635873=12, srv1711666545=37, srv835601345=91, srv1243037986=5, srv695595839=86, srv2058889191=57, srv170419557=36, srv1433184464=15, srv1233961599=4, srv1483325025=18, srv381383328=70, srv416102268=72, srv1725636586=38, srv720737640=88, srv1615391159=27, srv1498043605=20, srv1308665848=7, srv1325856009=9, srv997810221=99, srv1961060608=53, srv1619473217=28, srv1907360494=47, srv2111853692=60, srv2128687845=63, srv627751161=81, srv303273802=67, srv1366129112=13, srv1545929142=24, srv1658904352=34, srv523767842=78, srv1761719017=41, srv1956568182=52, srv402918819=71, srv1345168571=11, srv2054036935=56, srv2113502721=61, srv1890659170=46, srv163313346=29, srv990213595=98, srv602548272=80, srv1333359074=10, srv233997827=64, srv765635822=89, srv270264953=66, srv1493094350=19, srv1864549855=44, srv1422129196=14, srv1653680871=32, srv1574099083=25, srv1992542366=54, srv1025396175=0, srv1579378104=26, srv1947686916=51, srv853299198=92, srv858338485=93, srv639088198=83, srv718997813=87, srv1475339532=17, srv896466484=94, srv353287067=69, srv1667460962=35, srv636512982=82, srv954893921=97, srv1542775964=23, srv1849361639=43, srv1746189849=40, srv1658177069=33, srv2124837565=62, srv658979175=85, srv485892899=76, srv269695012=65, srv1807623869=42, srv1635477986=30, srv2092292682=59, srv1168670711=3, srv1082439299=2, srv1878988154=45, srv927274603=96, srv536556399=79, srv518853731=77, srv782423500=90, srv439259402=74} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:12,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:12,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:12,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:12,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-02 16:53:12,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-02 16:53:12,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-02 16:53:12,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-02 16:53:12,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-02 16:53:12,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-02 16:53:12,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-02 16:53:12,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-02 16:53:13,118 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.530647563026267E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:13,118 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1333359074:83 , srv1746189849:83 , srv627751161:83 , srv1025396175:84 , srv1040062295:84 , srv1082439299:84 , srv1168670711:84 , srv1233961599:84 , srv1243037986:84 , srv1261858336:84 , srv1322310512:84 , srv1325856009:84 , srv1345168571:84 , srv1354635873:84 , srv1366129112:84 , srv1433184464:84 , srv1455020546:84 , srv1475339532:84 , srv1483325025:84 , srv1493094350:84 , srv1498043605:84 , srv1517716592:84 , srv1521540967:84 , srv1542775964:84 , srv1545929142:84 , srv1574099083:84 , srv1579378104:84 , srv1619473217:84 , srv163313346:84 , srv1635477986:84 , srv1650135965:84 , srv1653680871:84 , srv1658177069:84 , srv1658904352:84 , srv1667460962:84 , srv170419557:84 , srv1711666545:84 , srv1725636586:84 , srv1738709781:84 , srv1761719017:84 , srv1807623869:84 , srv1849361639:84 , srv1864549855:84 , srv1878988154:84 , srv1890659170:84 , srv1907360494:84 , srv1917649018:84 , srv193498913:84 , srv1941119707:84 , srv1947686916:84 , srv1956568182:84 , srv1961060608:84 , srv1992542366:84 , srv1998247394:84 , srv2054036935:84 , srv2058889191:84 , srv2064051005:84 , srv2092292682:84 , srv2111853692:84 , srv2113502721:84 , srv2128687845:84 , srv233997827:84 , srv269695012:84 , srv270264953:84 , srv303273802:84 , srv329629260:84 , srv353287067:84 , srv381383328:84 , srv402918819:84 , srv416102268:84 , srv418271253:84 , srv439259402:84 , srv473461290:84 , srv485892899:84 , srv518853731:84 , srv523767842:84 , srv536556399:84 , srv602548272:84 , srv636512982:84 , srv639088198:84 , srv651455118:84 , srv658979175:84 , srv695595839:84 , srv718997813:84 , srv720737640:84 , srv765635822:84 , srv782423500:84 , srv835601345:84 , srv853299198:84 , srv858338485:84 , srv896466484:84 , srv919463122:84 , srv927274603:84 , srv954893921:84 , srv990213595:84 , srv997810221:84 , srv1308665848:85 , srv1422129196:85 , srv2124837565:85 , srv1615391159:684 }
2023-09-02 16:53:13,163 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=278 (was 278), ProcessCount=173 (was 173), AvailableMemoryMB=9357 (was 10221)
2023-09-02 16:53:13,177 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=278, ProcessCount=173, AvailableMemoryMB=9356
2023-09-02 16:53:13,177 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-02 16:53:13,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-02 16:53:13,190 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1026947080:84 , srv1058217266:84 , srv1102915110:84 , srv1133863097:84 , srv1173356898:84 , srv1439620185:84 , srv1566304360:84 , srv1591743703:84 , srv159466402:84 , srv1603721887:84 , srv1613714226:84 , srv1638090408:84 , srv1643864096:84 , srv1772482764:84 , srv1826629487:84 , srv2058718436:84 , srv207048575:84 , srv2113488358:84 , srv251173996:84 , srv326926479:84 , srv437383766:84 , srv52371784:84 , srv655542836:84 , srv678591177:84 , srv754971244:84 , srv784313754:84 , srv824583030:84 , srv908435747:84 , srv975974947:84 , srv659777008:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-02 16:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv159466402=8, srv1603721887=9, srv1439620185=5, srv754971244=25, srv1058217266=1, srv2113488358=17, srv975974947=29, srv1026947080=0, srv1826629487=14, srv659777008=23, srv1133863097=3, srv326926479=19, srv824583030=27, srv1772482764=13, srv655542836=22, srv1566304360=6, srv2058718436=15, srv1102915110=2, srv1173356898=4, srv1613714226=10, srv437383766=20, srv1591743703=7, srv251173996=18, srv1643864096=12, srv207048575=16, srv1638090408=11, srv52371784=21, srv678591177=24, srv784313754=26, srv908435747=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:53:13,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:53:13,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:53:13,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:53:13,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:53:13,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:53:13,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:53:13,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:53:13,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-02 16:53:13,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0652173281048816, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:53:13,227 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912398540935597, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0652173281048816, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-02 16:55:13,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5477434 different iterations.  Found a solution that moves 1865 regions; Going from a computed imbalance of 0.9912398540935597 to a new imbalance of 4.353003038599517E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6907407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:55:13,222 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1026947080:90 , srv1058217266:90 , srv1102915110:90 , srv1133863097:90 , srv1173356898:90 , srv1439620185:90 , srv1566304360:90 , srv1591743703:90 , srv159466402:90 , srv1603721887:90 , srv1613714226:90 , srv1638090408:90 , srv1643864096:90 , srv1772482764:90 , srv1826629487:90 , srv2058718436:90 , srv207048575:90 , srv2113488358:90 , srv251173996:90 , srv326926479:90 , srv437383766:90 , srv52371784:90 , srv655542836:90 , srv659777008:90 , srv678591177:90 , srv754971244:90 , srv784313754:90 , srv824583030:90 , srv908435747:90 , srv975974947:90 }
2023-09-02 16:55:13,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-02 16:55:13,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv159466402=8, srv1603721887=9, srv1439620185=5, srv754971244=25, srv1058217266=1, srv2113488358=17, srv975974947=29, srv1026947080=0, srv1826629487=14, srv659777008=23, srv1133863097=3, srv326926479=19, srv824583030=27, srv1772482764=13, srv655542836=22, srv1566304360=6, srv2058718436=15, srv1102915110=2, srv1173356898=4, srv1613714226=10, srv437383766=20, srv1591743703=7, srv251173996=18, srv1643864096=12, srv207048575=16, srv1638090408=11, srv52371784=21, srv678591177=24, srv784313754=26, srv908435747=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-02 16:55:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-02 16:55:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-02 16:55:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-02 16:55:13,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-02 16:55:13,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-02 16:55:13,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-02 16:55:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-02 16:55:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-02 16:55:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-02 16:55:13,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-02 16:55:13,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-02 16:55:13,250 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1026947080:90 , srv1058217266:90 , srv1102915110:90 , srv1133863097:90 , srv1173356898:90 , srv1439620185:90 , srv1566304360:90 , srv1591743703:90 , srv159466402:90 , srv1603721887:90 , srv1613714226:90 , srv1638090408:90 , srv1643864096:90 , srv1772482764:90 , srv1826629487:90 , srv2058718436:90 , srv207048575:90 , srv2113488358:90 , srv251173996:90 , srv326926479:90 , srv437383766:90 , srv52371784:90 , srv655542836:90 , srv659777008:90 , srv678591177:90 , srv754971244:90 , srv784313754:90 , srv824583030:90 , srv908435747:90 , srv975974947:90 }
2023-09-02 16:55:13,267 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=178 (was 278), ProcessCount=168 (was 173), AvailableMemoryMB=9293 (was 9356)