2023-12-08 14:56:54,530 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-08 14:56:54,893 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-08 14:56:55,084 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-08 14:56:55,192 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-08 14:56:55,205 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-08 14:56:55,231 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=276, ProcessCount=172, AvailableMemoryMB=1456
2023-12-08 14:56:55,233 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-08 14:56:55,234 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-08 14:56:55,416 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1066518527:84 , srv1072065140:84 , srv1083556878:84 , srv1090141453:84 , srv1093071850:84 , srv1164822921:84 , srv1199973644:84 , srv120322240:84 , srv1211697208:84 , srv1217200230:84 , srv1219903810:84 , srv1293757117:84 , srv1298718629:84 , srv1358305937:84 , srv1446273439:84 , srv1448127984:84 , srv145824674:84 , srv146783396:84 , srv1488466748:84 , srv153225327:84 , srv1543975732:84 , srv1575594490:84 , srv1609039227:84 , srv160952207:84 , srv1626715263:84 , srv1630768800:84 , srv1653035242:84 , srv1662151765:84 , srv1665878028:84 , srv1691295152:84 , srv1782082236:84 , srv1844498020:84 , srv1846132529:84 , srv1866130856:84 , srv1894983617:84 , srv1908799166:84 , srv1950591569:84 , srv1990634594:84 , srv1994610247:84 , srv2017353987:84 , srv204412564:84 , srv2044700641:84 , srv2092325942:84 , srv2093539826:84 , srv2113414736:84 , srv2114453396:84 , srv2125936490:84 , srv213556830:84 , srv2144683807:84 , srv244911584:84 , srv245898377:84 , srv259233074:84 , srv27190090:84 , srv298555665:84 , srv302831507:84 , srv304260392:84 , srv313954951:84 , srv350269685:84 , srv35639120:84 , srv359702865:84 , srv384085045:84 , srv386632747:84 , srv393427427:84 , srv40326675:84 , srv407904176:84 , srv434411460:84 , srv472965907:84 , srv496540891:84 , srv497866408:84 , srv504174093:84 , srv511877927:84 , srv513260399:84 , srv585068019:84 , srv586426381:84 , srv586597108:84 , srv613614357:84 , srv622293425:84 , srv625407521:84 , srv629592046:84 , srv630524842:84 , srv634548726:84 , srv667989746:84 , srv675624199:84 , srv693952664:84 , srv696087835:84 , srv704855665:84 , srv763121341:84 , srv770234697:84 , srv806291384:84 , srv821482797:84 , srv829294655:84 , srv859909057:84 , srv869199822:84 , srv871288773:84 , srv871443241:84 , srv916533696:84 , srv984773772:84 , srv993973083:84 , srv997634102:84 , srv2096825888:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-08 14:56:55,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:55,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv204412564=40, srv704855665=86, srv2096825888=44, srv359702865=60, srv859909057=92, srv1083556878=2, srv585068019=73, srv1866130856=33, srv472965907=67, srv146783396=17, srv1665878028=28, srv1298718629=12, srv1488466748=18, srv1066518527=0, srv1093071850=4, srv1448127984=15, srv40326675=64, srv625407521=78, srv160952207=23, srv696087835=85, srv384085045=61, srv2144683807=49, srv2092325942=42, srv2017353987=39, srv2125936490=47, srv434411460=66, srv1662151765=27, srv916533696=96, srv997634102=99, srv497866408=69, srv1894983617=34, srv1164822921=5, srv1950591569=36, srv513260399=72, srv245898377=51, srv2113414736=45, srv1609039227=22, srv298555665=54, srv675624199=83, srv1217200230=9, srv1219903810=10, srv1358305937=13, srv1846132529=32, srv1908799166=35, srv869199822=93, srv302831507=55, srv407904176=65, srv586426381=74, srv1211697208=8, srv1626715263=24, srv984773772=97, srv667989746=82, srv586597108=75, srv613614357=76, srv1630768800=25, srv259233074=52, srv871443241=95, srv634548726=81, srv1072065140=1, srv153225327=19, srv386632747=62, srv1543975732=20, srv1575594490=21, srv821482797=90, srv393427427=63, srv213556830=48, srv871288773=94, srv2114453396=46, srv244911584=50, srv2093539826=43, srv504174093=70, srv1446273439=14, srv304260392=56, srv806291384=89, srv496540891=68, srv350269685=58, srv1990634594=37, srv829294655=91, srv1691295152=29, srv763121341=87, srv313954951=57, srv145824674=16, srv629592046=79, srv511877927=71, srv693952664=84, srv770234697=88, srv622293425=77, srv630524842=80, srv1199973644=6, srv993973083=98, srv120322240=7, srv1782082236=30, srv1653035242=26, srv2044700641=41, srv1090141453=3, srv1994610247=38, srv1844498020=31, srv35639120=59, srv1293757117=11, srv27190090=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:55,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:55,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:55,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:55,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:55,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:55,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:55,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:55,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:55,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:55,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:55,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-08 14:56:55,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-08 14:56:55,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-08 14:56:55,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-08 14:56:55,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-08 14:56:55,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-08 14:56:55,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-08 14:56:55,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-08 14:56:55,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-08 14:56:55,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-08 14:56:55,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-08 14:56:55,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-08 14:56:55,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-08 14:56:55,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-08 14:56:55,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-08 14:56:55,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-08 14:56:55,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-08 14:56:55,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-08 14:56:55,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-08 14:56:55,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-08 14:56:55,943 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:55,943 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-08 14:56:55,944 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-08 14:56:56,490 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 868 ms to try 5000 different iterations.  Found a solution that moves 4699 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17497246634169664. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03759843878178699, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051626735954540696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5221111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03759843878178699, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16991666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:56,512 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1217200230:83 , srv1990634594:83 , srv2093539826:83 , srv27190090:83 , srv302831507:83 , srv667989746:83 , srv160952207:84 , srv1662151765:84 , srv1844498020:84 , srv1846132529:84 , srv213556830:84 , srv35639120:84 , srv386632747:84 , srv497866408:84 , srv504174093:84 , srv585068019:84 , srv675624199:84 , srv806291384:84 , srv871288773:84 , srv984773772:84 , srv997634102:84 , srv1164822921:85 , srv120322240:85 , srv1293757117:85 , srv1630768800:85 , srv1653035242:85 , srv1950591569:85 , srv204412564:85 , srv244911584:85 , srv384085045:85 , srv630524842:85 , srv763121341:85 , srv770234697:85 , srv993973083:85 , srv1083556878:86 , srv1199973644:86 , srv1446273439:86 , srv1488466748:86 , srv1575594490:86 , srv1665878028:86 , srv1782082236:86 , srv1894983617:86 , srv1994610247:86 , srv2092325942:86 , srv2114453396:86 , srv2125936490:86 , srv298555665:86 , srv40326675:86 , srv586597108:86 , srv634548726:86 , srv693952664:86 , srv696087835:86 , srv829294655:86 , srv916533696:86 , srv1298718629:87 , srv1448127984:87 , srv1543975732:87 , srv1691295152:87 , srv1866130856:87 , srv304260392:87 , srv359702865:87 , srv393427427:87 , srv511877927:87 , srv622293425:87 , srv704855665:87 , srv859909057:87 , srv1090141453:88 , srv1211697208:88 , srv1358305937:88 , srv145824674:88 , srv146783396:88 , srv1908799166:88 , srv2017353987:88 , srv2044700641:88 , srv2113414736:88 , srv2144683807:88 , srv259233074:88 , srv350269685:88 , srv407904176:88 , srv434411460:88 , srv472965907:88 , srv513260399:88 , srv821482797:88 , srv1072065140:89 , srv1609039227:89 , srv245898377:89 , srv313954951:89 , srv496540891:89 , srv625407521:89 , srv871443241:89 , srv1219903810:90 , srv586426381:90 , srv613614357:90 , srv869199822:90 , srv1093071850:91 , srv153225327:91 , srv629592046:92 , srv1626715263:96 , srv1066518527:100 , srv2096825888:424 }
2023-12-08 14:56:56,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:56,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv204412564=40, srv704855665=86, srv2096825888=44, srv359702865=60, srv859909057=92, srv1083556878=2, srv585068019=73, srv1866130856=33, srv472965907=67, srv146783396=17, srv1665878028=28, srv1298718629=12, srv1488466748=18, srv1066518527=0, srv1093071850=4, srv1448127984=15, srv40326675=64, srv625407521=78, srv160952207=23, srv696087835=85, srv384085045=61, srv2144683807=49, srv2092325942=42, srv2017353987=39, srv2125936490=47, srv434411460=66, srv1662151765=27, srv916533696=96, srv997634102=99, srv497866408=69, srv1894983617=34, srv1164822921=5, srv1950591569=36, srv513260399=72, srv245898377=51, srv2113414736=45, srv1609039227=22, srv298555665=54, srv675624199=83, srv1217200230=9, srv1219903810=10, srv1358305937=13, srv1846132529=32, srv1908799166=35, srv869199822=93, srv302831507=55, srv407904176=65, srv586426381=74, srv1211697208=8, srv1626715263=24, srv984773772=97, srv667989746=82, srv586597108=75, srv613614357=76, srv1630768800=25, srv259233074=52, srv871443241=95, srv634548726=81, srv1072065140=1, srv153225327=19, srv386632747=62, srv1543975732=20, srv1575594490=21, srv821482797=90, srv393427427=63, srv213556830=48, srv871288773=94, srv2114453396=46, srv244911584=50, srv2093539826=43, srv504174093=70, srv1446273439=14, srv304260392=56, srv806291384=89, srv496540891=68, srv350269685=58, srv1990634594=37, srv829294655=91, srv1691295152=29, srv763121341=87, srv313954951=57, srv145824674=16, srv629592046=79, srv511877927=71, srv693952664=84, srv770234697=88, srv622293425=77, srv630524842=80, srv1199973644=6, srv993973083=98, srv120322240=7, srv1782082236=30, srv1653035242=26, srv2044700641=41, srv1090141453=3, srv1994610247=38, srv1844498020=31, srv35639120=59, srv1293757117=11, srv27190090=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:56,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:56,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:56,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:56,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:56,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:56,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:56,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:56,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:56,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:56,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:56,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:56,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:56,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:56,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-08 14:56:56,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-08 14:56:56,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-08 14:56:56,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-08 14:56:56,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-08 14:56:56,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-08 14:56:56,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-08 14:56:56,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-08 14:56:56,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-08 14:56:56,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-08 14:56:56,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-08 14:56:56,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-08 14:56:56,780 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03759843878178699, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051626735954540696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03759843878178699, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16991666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:56,780 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-08 14:56:56,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17493956324044455, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03759843878178699, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051626735954540696, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03759843878178699, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16991666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23941666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-08 14:56:57,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 416 ms to try 5000 different iterations.  Found a solution that moves 2672 regions; Going from a computed imbalance of 0.17493956324044455 to a new imbalance of 0.015921684810360614. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02103210451018152); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02707183775717981, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29688888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02103210451018152); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:57,061 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1066518527:72 , srv1626715263:76 , srv153225327:81 , srv586426381:81 , srv871443241:81 , srv1072065140:82 , srv1093071850:82 , srv1219903810:82 , srv245898377:82 , srv629592046:82 , srv1211697208:83 , srv1609039227:83 , srv2044700641:83 , srv2113414736:83 , srv304260392:83 , srv313954951:83 , srv407904176:83 , srv613614357:83 , srv821482797:83 , srv869199822:83 , srv1446273439:84 , srv145824674:84 , srv1543975732:84 , srv1908799166:84 , srv259233074:84 , srv350269685:84 , srv513260399:84 , srv859909057:84 , srv1090141453:85 , srv1298718629:85 , srv1358305937:85 , srv1488466748:85 , srv1575594490:85 , srv1653035242:85 , srv1665878028:85 , srv1782082236:85 , srv1894983617:85 , srv1994610247:85 , srv2114453396:85 , srv359702865:85 , srv384085045:85 , srv434411460:85 , srv496540891:85 , srv625407521:85 , srv704855665:85 , srv763121341:85 , srv916533696:85 , srv1083556878:86 , srv1199973644:86 , srv1293757117:86 , srv146783396:86 , srv1630768800:86 , srv1691295152:86 , srv1844498020:86 , srv1950591569:86 , srv2017353987:86 , srv2092325942:86 , srv2144683807:86 , srv298555665:86 , srv393427427:86 , srv504174093:86 , srv511877927:86 , srv693952664:86 , srv993973083:86 , srv120322240:87 , srv1448127984:87 , srv160952207:87 , srv1866130856:87 , srv204412564:87 , srv244911584:87 , srv472965907:87 , srv585068019:87 , srv622293425:87 , srv630524842:87 , srv634548726:87 , srv829294655:87 , srv1217200230:88 , srv1846132529:88 , srv2125936490:88 , srv27190090:88 , srv386632747:88 , srv40326675:88 , srv497866408:88 , srv586597108:88 , srv696087835:88 , srv997634102:88 , srv1164822921:89 , srv1662151765:89 , srv1990634594:89 , srv2093539826:89 , srv213556830:89 , srv35639120:89 , srv667989746:89 , srv675624199:89 , srv806291384:89 , srv871288773:89 , srv984773772:89 , srv302831507:90 , srv770234697:90 , srv2096825888:537 }
2023-12-08 14:56:57,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:57,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv204412564=40, srv704855665=86, srv2096825888=44, srv359702865=60, srv859909057=92, srv1083556878=2, srv585068019=73, srv1866130856=33, srv472965907=67, srv146783396=17, srv1665878028=28, srv1298718629=12, srv1488466748=18, srv1066518527=0, srv1093071850=4, srv1448127984=15, srv40326675=64, srv625407521=78, srv160952207=23, srv696087835=85, srv384085045=61, srv2144683807=49, srv2092325942=42, srv2017353987=39, srv2125936490=47, srv434411460=66, srv1662151765=27, srv916533696=96, srv997634102=99, srv497866408=69, srv1894983617=34, srv1164822921=5, srv1950591569=36, srv513260399=72, srv245898377=51, srv2113414736=45, srv1609039227=22, srv298555665=54, srv675624199=83, srv1217200230=9, srv1219903810=10, srv1358305937=13, srv1846132529=32, srv1908799166=35, srv869199822=93, srv302831507=55, srv407904176=65, srv586426381=74, srv1211697208=8, srv1626715263=24, srv984773772=97, srv667989746=82, srv586597108=75, srv613614357=76, srv1630768800=25, srv259233074=52, srv871443241=95, srv634548726=81, srv1072065140=1, srv153225327=19, srv386632747=62, srv1543975732=20, srv1575594490=21, srv821482797=90, srv393427427=63, srv213556830=48, srv871288773=94, srv2114453396=46, srv244911584=50, srv2093539826=43, srv504174093=70, srv1446273439=14, srv304260392=56, srv806291384=89, srv496540891=68, srv350269685=58, srv1990634594=37, srv829294655=91, srv1691295152=29, srv763121341=87, srv313954951=57, srv145824674=16, srv629592046=79, srv511877927=71, srv693952664=84, srv770234697=88, srv622293425=77, srv630524842=80, srv1199973644=6, srv993973083=98, srv120322240=7, srv1782082236=30, srv1653035242=26, srv2044700641=41, srv1090141453=3, srv1994610247=38, srv1844498020=31, srv35639120=59, srv1293757117=11, srv27190090=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:57,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:57,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:57,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:57,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:57,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-08 14:56:57,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-08 14:56:57,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-08 14:56:57,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-08 14:56:57,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-08 14:56:57,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-08 14:56:57,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-08 14:56:57,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-08 14:56:57,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-08 14:56:57,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-08 14:56:57,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-08 14:56:57,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-08 14:56:57,251 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02103210451018152); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02707183775717981, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02103210451018152); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:57,251 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-08 14:56:57,252 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01590297506646924, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02103210451018152); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02707183775717981, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02103210451018152); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-08 14:56:57,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 981 regions; Going from a computed imbalance of 0.01590297506646924 to a new imbalance of 7.942603129069548E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006315083873134922); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008164965809277261); MoveCostFunction : (multiplier=7.0, imbalance=0.109, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006315083873134922); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:57,586 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv472965907:82 , srv770234697:82 , srv1164822921:83 , srv146783396:83 , srv1866130856:83 , srv2017353987:83 , srv2144683807:83 , srv40326675:83 , srv496540891:83 , srv586597108:83 , srv696087835:83 , srv1090141453:84 , srv1093071850:84 , srv1358305937:84 , srv1448127984:84 , srv1662151765:84 , srv2125936490:84 , srv213556830:84 , srv302831507:84 , srv35639120:84 , srv613614357:84 , srv622293425:84 , srv625407521:84 , srv629592046:84 , srv634548726:84 , srv806291384:84 , srv829294655:84 , srv984773772:84 , srv1083556878:85 , srv120322240:85 , srv1298718629:85 , srv145824674:85 , srv1626715263:85 , srv1630768800:85 , srv1691295152:85 , srv1846132529:85 , srv1908799166:85 , srv204412564:85 , srv359702865:85 , srv393427427:85 , srv434411460:85 , srv497866408:85 , srv511877927:85 , srv513260399:85 , srv675624199:85 , srv693952664:85 , srv869199822:85 , srv871288773:85 , srv997634102:85 , srv1066518527:86 , srv1072065140:86 , srv1199973644:86 , srv1217200230:86 , srv1293757117:86 , srv153225327:86 , srv1543975732:86 , srv1575594490:86 , srv1609039227:86 , srv1894983617:86 , srv1990634594:86 , srv1994610247:86 , srv2092325942:86 , srv2093539826:86 , srv2113414736:86 , srv244911584:86 , srv27190090:86 , srv298555665:86 , srv313954951:86 , srv350269685:86 , srv386632747:86 , srv407904176:86 , srv585068019:86 , srv630524842:86 , srv667989746:86 , srv704855665:86 , srv821482797:86 , srv1211697208:87 , srv1219903810:87 , srv1446273439:87 , srv1488466748:87 , srv160952207:87 , srv1653035242:87 , srv1665878028:87 , srv1782082236:87 , srv1844498020:87 , srv1950591569:87 , srv2044700641:87 , srv2114453396:87 , srv245898377:87 , srv259233074:87 , srv304260392:87 , srv384085045:87 , srv504174093:87 , srv586426381:87 , srv763121341:87 , srv859909057:87 , srv871443241:87 , srv916533696:87 , srv993973083:87 , srv2096825888:553 }
2023-12-08 14:56:57,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:57,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv204412564=40, srv704855665=86, srv2096825888=44, srv359702865=60, srv859909057=92, srv1083556878=2, srv585068019=73, srv1866130856=33, srv472965907=67, srv146783396=17, srv1665878028=28, srv1298718629=12, srv1488466748=18, srv1066518527=0, srv1093071850=4, srv1448127984=15, srv40326675=64, srv625407521=78, srv160952207=23, srv696087835=85, srv384085045=61, srv2144683807=49, srv2092325942=42, srv2017353987=39, srv2125936490=47, srv434411460=66, srv1662151765=27, srv916533696=96, srv997634102=99, srv497866408=69, srv1894983617=34, srv1164822921=5, srv1950591569=36, srv513260399=72, srv245898377=51, srv2113414736=45, srv1609039227=22, srv298555665=54, srv675624199=83, srv1217200230=9, srv1219903810=10, srv1358305937=13, srv1846132529=32, srv1908799166=35, srv869199822=93, srv302831507=55, srv407904176=65, srv586426381=74, srv1211697208=8, srv1626715263=24, srv984773772=97, srv667989746=82, srv586597108=75, srv613614357=76, srv1630768800=25, srv259233074=52, srv871443241=95, srv634548726=81, srv1072065140=1, srv153225327=19, srv386632747=62, srv1543975732=20, srv1575594490=21, srv821482797=90, srv393427427=63, srv213556830=48, srv871288773=94, srv2114453396=46, srv244911584=50, srv2093539826=43, srv504174093=70, srv1446273439=14, srv304260392=56, srv806291384=89, srv496540891=68, srv350269685=58, srv1990634594=37, srv829294655=91, srv1691295152=29, srv763121341=87, srv313954951=57, srv145824674=16, srv629592046=79, srv511877927=71, srv693952664=84, srv770234697=88, srv622293425=77, srv630524842=80, srv1199973644=6, srv993973083=98, srv120322240=7, srv1782082236=30, srv1653035242=26, srv2044700641=41, srv1090141453=3, srv1994610247=38, srv1844498020=31, srv35639120=59, srv1293757117=11, srv27190090=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:57,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:57,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:57,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:57,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:57,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:57,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:57,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-08 14:56:57,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-08 14:56:57,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-08 14:56:57,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-08 14:56:57,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-08 14:56:57,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-08 14:56:57,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-08 14:56:57,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-08 14:56:57,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-08 14:56:57,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-08 14:56:57,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-08 14:56:57,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-08 14:56:57,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-08 14:56:57,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-08 14:56:57,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-08 14:56:57,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-08 14:56:57,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-08 14:56:57,722 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006315083873134922); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008164965809277261); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006315083873134922); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:57,723 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-08 14:56:57,723 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.873912040905482E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006315083873134922); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008164965809277261); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006315083873134922); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-08 14:56:57,978 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 339 ms to try 5000 different iterations.  Found a solution that moves 250 regions; Going from a computed imbalance of 7.873912040905482E-4 to a new imbalance of 1.2676210020363922E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=6.116465361813114E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.027777777777777776, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=6.116465361813114E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:57,979 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1066518527:83 , srv1219903810:83 , srv1665878028:83 , srv1846132529:83 , srv1950591569:83 , srv2114453396:83 , srv259233074:83 , srv1199973644:84 , srv1211697208:84 , srv1298718629:84 , srv1448127984:84 , srv1488466748:84 , srv153225327:84 , srv160952207:84 , srv1626715263:84 , srv1691295152:84 , srv1990634594:84 , srv2044700641:84 , srv2092325942:84 , srv2093539826:84 , srv2125936490:84 , srv213556830:84 , srv245898377:84 , srv313954951:84 , srv386632747:84 , srv393427427:84 , srv497866408:84 , srv513260399:84 , srv586426381:84 , srv613614357:84 , srv622293425:84 , srv625407521:84 , srv629592046:84 , srv630524842:84 , srv675624199:84 , srv704855665:84 , srv770234697:84 , srv829294655:84 , srv859909057:84 , srv869199822:84 , srv871288773:84 , srv916533696:84 , srv993973083:84 , srv1072065140:85 , srv1083556878:85 , srv1090141453:85 , srv1093071850:85 , srv120322240:85 , srv1217200230:85 , srv1293757117:85 , srv1358305937:85 , srv1446273439:85 , srv145824674:85 , srv146783396:85 , srv1543975732:85 , srv1575594490:85 , srv1609039227:85 , srv1662151765:85 , srv1782082236:85 , srv1844498020:85 , srv1866130856:85 , srv1894983617:85 , srv1908799166:85 , srv1994610247:85 , srv2017353987:85 , srv204412564:85 , srv2113414736:85 , srv2144683807:85 , srv244911584:85 , srv27190090:85 , srv298555665:85 , srv302831507:85 , srv304260392:85 , srv350269685:85 , srv359702865:85 , srv384085045:85 , srv40326675:85 , srv407904176:85 , srv434411460:85 , srv472965907:85 , srv496540891:85 , srv504174093:85 , srv511877927:85 , srv585068019:85 , srv586597108:85 , srv634548726:85 , srv667989746:85 , srv693952664:85 , srv763121341:85 , srv871443241:85 , srv984773772:85 , srv997634102:85 , srv1164822921:86 , srv1630768800:86 , srv1653035242:86 , srv35639120:86 , srv696087835:86 , srv806291384:86 , srv821482797:86 , srv2096825888:628 }
2023-12-08 14:56:57,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:57,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv204412564=40, srv704855665=86, srv2096825888=44, srv359702865=60, srv859909057=92, srv1083556878=2, srv585068019=73, srv1866130856=33, srv472965907=67, srv146783396=17, srv1665878028=28, srv1298718629=12, srv1488466748=18, srv1066518527=0, srv1093071850=4, srv1448127984=15, srv40326675=64, srv625407521=78, srv160952207=23, srv696087835=85, srv384085045=61, srv2144683807=49, srv2092325942=42, srv2017353987=39, srv2125936490=47, srv434411460=66, srv1662151765=27, srv916533696=96, srv997634102=99, srv497866408=69, srv1894983617=34, srv1164822921=5, srv1950591569=36, srv513260399=72, srv245898377=51, srv2113414736=45, srv1609039227=22, srv298555665=54, srv675624199=83, srv1217200230=9, srv1219903810=10, srv1358305937=13, srv1846132529=32, srv1908799166=35, srv869199822=93, srv302831507=55, srv407904176=65, srv586426381=74, srv1211697208=8, srv1626715263=24, srv984773772=97, srv667989746=82, srv586597108=75, srv613614357=76, srv1630768800=25, srv259233074=52, srv871443241=95, srv634548726=81, srv1072065140=1, srv153225327=19, srv386632747=62, srv1543975732=20, srv1575594490=21, srv821482797=90, srv393427427=63, srv213556830=48, srv871288773=94, srv2114453396=46, srv244911584=50, srv2093539826=43, srv504174093=70, srv1446273439=14, srv304260392=56, srv806291384=89, srv496540891=68, srv350269685=58, srv1990634594=37, srv829294655=91, srv1691295152=29, srv763121341=87, srv313954951=57, srv145824674=16, srv629592046=79, srv511877927=71, srv693952664=84, srv770234697=88, srv622293425=77, srv630524842=80, srv1199973644=6, srv993973083=98, srv120322240=7, srv1782082236=30, srv1653035242=26, srv2044700641=41, srv1090141453=3, srv1994610247=38, srv1844498020=31, srv35639120=59, srv1293757117=11, srv27190090=53} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:58,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:58,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:58,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:58,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:58,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:58,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-08 14:56:58,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-08 14:56:58,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-08 14:56:58,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-08 14:56:58,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-08 14:56:58,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-08 14:56:58,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-08 14:56:58,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-08 14:56:58,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-08 14:56:58,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-08 14:56:58,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-08 14:56:58,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-08 14:56:58,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-08 14:56:58,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0925672605377522E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=6.116465361813114E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=6.116465361813114E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:58,128 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1066518527:83 , srv1219903810:83 , srv1665878028:83 , srv1846132529:83 , srv1950591569:83 , srv2114453396:83 , srv259233074:83 , srv1199973644:84 , srv1211697208:84 , srv1298718629:84 , srv1448127984:84 , srv1488466748:84 , srv153225327:84 , srv160952207:84 , srv1626715263:84 , srv1691295152:84 , srv1990634594:84 , srv2044700641:84 , srv2092325942:84 , srv2093539826:84 , srv2125936490:84 , srv213556830:84 , srv245898377:84 , srv313954951:84 , srv386632747:84 , srv393427427:84 , srv497866408:84 , srv513260399:84 , srv586426381:84 , srv613614357:84 , srv622293425:84 , srv625407521:84 , srv629592046:84 , srv630524842:84 , srv675624199:84 , srv704855665:84 , srv770234697:84 , srv829294655:84 , srv859909057:84 , srv869199822:84 , srv871288773:84 , srv916533696:84 , srv993973083:84 , srv1072065140:85 , srv1083556878:85 , srv1090141453:85 , srv1093071850:85 , srv120322240:85 , srv1217200230:85 , srv1293757117:85 , srv1358305937:85 , srv1446273439:85 , srv145824674:85 , srv146783396:85 , srv1543975732:85 , srv1575594490:85 , srv1609039227:85 , srv1662151765:85 , srv1782082236:85 , srv1844498020:85 , srv1866130856:85 , srv1894983617:85 , srv1908799166:85 , srv1994610247:85 , srv2017353987:85 , srv204412564:85 , srv2113414736:85 , srv2144683807:85 , srv244911584:85 , srv27190090:85 , srv298555665:85 , srv302831507:85 , srv304260392:85 , srv350269685:85 , srv359702865:85 , srv384085045:85 , srv40326675:85 , srv407904176:85 , srv434411460:85 , srv472965907:85 , srv496540891:85 , srv504174093:85 , srv511877927:85 , srv585068019:85 , srv586597108:85 , srv634548726:85 , srv667989746:85 , srv693952664:85 , srv763121341:85 , srv871443241:85 , srv984773772:85 , srv997634102:85 , srv1164822921:86 , srv1630768800:86 , srv1653035242:86 , srv35639120:86 , srv696087835:86 , srv806291384:86 , srv821482797:86 , srv2096825888:628 }
2023-12-08 14:56:58,171 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=382 (was 276) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 172), AvailableMemoryMB=964 (was 1456)
2023-12-08 14:56:58,183 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=292, MaxFileDescriptor=60000, SystemLoadAverage=382, ProcessCount=170, AvailableMemoryMB=964
2023-12-08 14:56:58,184 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-08 14:56:58,184 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-08 14:56:58,196 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv106870457:84 , srv1114259130:84 , srv1192681088:84 , srv1402184031:84 , srv1414973153:84 , srv1472064533:84 , srv1474881483:84 , srv1509400000:84 , srv1597171144:84 , srv1638464773:84 , srv1785925658:84 , srv1836046102:84 , srv2034042397:84 , srv2111418774:84 , srv2142252289:84 , srv273133896:84 , srv28899104:84 , srv362975646:84 , srv388674540:84 , srv611251906:84 , srv666893787:84 , srv690425330:84 , srv69291139:84 , srv706427543:84 , srv743848044:84 , srv749529148:84 , srv867282188:84 , srv938389027:84 , srv99974805:84 , srv1729281966:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-08 14:56:58,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:56:58,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1414973153=4, srv1638464773=9, srv273133896=16, srv743848044=25, srv666893787=21, srv106870457=0, srv867282188=27, srv69291139=23, srv1114259130=1, srv1402184031=3, srv1597171144=8, srv706427543=24, srv1785925658=11, srv611251906=20, srv388674540=19, srv749529148=26, srv1474881483=6, srv938389027=28, srv2142252289=15, srv362975646=18, srv1192681088=2, srv2111418774=14, srv1509400000=7, srv2034042397=13, srv1729281966=10, srv1472064533=5, srv1836046102=12, srv28899104=17, srv99974805=29, srv690425330=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:56:58,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:56:58,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:56:58,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:56:58,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-08 14:56:58,240 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1599320769899442, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:56:58,241 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912696984016611, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1599320769899442, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-08 14:58:58,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9679983 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.9912696984016611 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:58:58,226 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv106870457:90 , srv1114259130:90 , srv1192681088:90 , srv1402184031:90 , srv1414973153:90 , srv1472064533:90 , srv1474881483:90 , srv1509400000:90 , srv1597171144:90 , srv1638464773:90 , srv1729281966:90 , srv1785925658:90 , srv1836046102:90 , srv2034042397:90 , srv2111418774:90 , srv2142252289:90 , srv273133896:90 , srv28899104:90 , srv362975646:90 , srv388674540:90 , srv611251906:90 , srv666893787:90 , srv690425330:90 , srv69291139:90 , srv706427543:90 , srv743848044:90 , srv749529148:90 , srv867282188:90 , srv938389027:90 , srv99974805:90 }
2023-12-08 14:58:58,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-08 14:58:58,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1414973153=4, srv1638464773=9, srv273133896=16, srv743848044=25, srv666893787=21, srv106870457=0, srv867282188=27, srv69291139=23, srv1114259130=1, srv1402184031=3, srv1597171144=8, srv706427543=24, srv1785925658=11, srv611251906=20, srv388674540=19, srv749529148=26, srv1474881483=6, srv938389027=28, srv2142252289=15, srv362975646=18, srv1192681088=2, srv2111418774=14, srv1509400000=7, srv2034042397=13, srv1729281966=10, srv1472064533=5, srv1836046102=12, srv28899104=17, srv99974805=29, srv690425330=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-08 14:58:58,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-08 14:58:58,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-08 14:58:58,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-08 14:58:58,241 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-08 14:58:58,241 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv106870457:90 , srv1114259130:90 , srv1192681088:90 , srv1402184031:90 , srv1414973153:90 , srv1472064533:90 , srv1474881483:90 , srv1509400000:90 , srv1597171144:90 , srv1638464773:90 , srv1729281966:90 , srv1785925658:90 , srv1836046102:90 , srv2034042397:90 , srv2111418774:90 , srv2142252289:90 , srv273133896:90 , srv28899104:90 , srv362975646:90 , srv388674540:90 , srv611251906:90 , srv666893787:90 , srv690425330:90 , srv69291139:90 , srv706427543:90 , srv743848044:90 , srv749529148:90 , srv867282188:90 , srv938389027:90 , srv99974805:90 }
2023-12-08 14:58:58,251 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 292), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=186 (was 382), ProcessCount=169 (was 170), AvailableMemoryMB=2119 (was 964) - AvailableMemoryMB LEAK? -