2023-09-24 04:53:15,754 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-24 04:53:16,084 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-24 04:53:16,196 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-24 04:53:16,260 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 04:53:16,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-24 04:53:16,285 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=173, AvailableMemoryMB=10080
2023-09-24 04:53:16,287 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 04:53:16,288 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-24 04:53:16,439 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv101750058:84 , srv1029595954:84 , srv1066914141:84 , srv1130249989:84 , srv1130812493:84 , srv1143111949:84 , srv1150685296:84 , srv1154402644:84 , srv1165670430:84 , srv1172115749:84 , srv1191128800:84 , srv1192029661:84 , srv119424825:84 , srv1242126737:84 , srv1258580679:84 , srv1260820477:84 , srv126976538:84 , srv1271158608:84 , srv1309690657:84 , srv1326241449:84 , srv13268465:84 , srv1353201827:84 , srv1398671422:84 , srv1406228504:84 , srv1411547296:84 , srv1420310179:84 , srv1434796053:84 , srv1485851064:84 , srv1498951008:84 , srv1509429456:84 , srv1518500164:84 , srv1524512316:84 , srv156001983:84 , srv1592761274:84 , srv1598472599:84 , srv1613433912:84 , srv1622732230:84 , srv1633961469:84 , srv1675767291:84 , srv1705281587:84 , srv1711001418:84 , srv1721319151:84 , srv1727460862:84 , srv1730963918:84 , srv1736366041:84 , srv1737795890:84 , srv1739853844:84 , srv1763814534:84 , srv1770863979:84 , srv1776247967:84 , srv1830858695:84 , srv186328237:84 , srv1870592288:84 , srv1871747099:84 , srv1895164977:84 , srv1920606680:84 , srv1963390247:84 , srv1981864639:84 , srv2013467195:84 , srv2100236306:84 , srv2125296306:84 , srv2140325979:84 , srv2145265884:84 , srv218130788:84 , srv244080472:84 , srv253774427:84 , srv26920791:84 , srv291424198:84 , srv295199016:84 , srv323490212:84 , srv32678487:84 , srv329010883:84 , srv334624726:84 , srv400457983:84 , srv44010105:84 , srv469854768:84 , srv47271895:84 , srv474981157:84 , srv48201877:84 , srv505596989:84 , srv537141589:84 , srv558087435:84 , srv598494785:84 , srv661740573:84 , srv707185847:84 , srv721339763:84 , srv74404835:84 , srv757624712:84 , srv763534872:84 , srv785658370:84 , srv803952363:84 , srv825718935:84 , srv826618282:84 , srv828846443:84 , srv876605774:84 , srv898687215:84 , srv91009691:84 , srv946829752:84 , srv958958202:84 , srv1797740108:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-24 04:53:16,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:16,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv119424825=12, srv1737795890=45, srv1598472599=34, srv1830858695=51, srv469854768=76, srv186328237=52, srv958958202=99, srv1258580679=14, srv1192029661=11, srv47271895=77, srv1326241449=19, srv48201877=79, srv1730963918=43, srv1191128800=10, srv1242126737=13, srv946829752=98, srv505596989=80, srv876605774=95, srv558087435=82, srv1411547296=24, srv1920606680=56, srv1776247967=49, srv334624726=73, srv32678487=71, srv1150685296=6, srv1727460862=42, srv1524512316=31, srv898687215=96, srv1420310179=25, srv707185847=85, srv2013467195=59, srv295199016=69, srv1981864639=58, srv101750058=0, srv474981157=78, srv1871747099=54, srv126976538=16, srv1721319151=41, srv803952363=91, srv1770863979=48, srv1592761274=33, srv1029595954=1, srv156001983=32, srv1613433912=35, srv1622732230=36, srv1066914141=2, srv1963390247=57, srv253774427=66, srv1130249989=3, srv1165670430=8, srv1705281587=39, srv44010105=75, srv1633961469=37, srv26920791=67, srv1154402644=7, srv1675767291=38, srv1498951008=28, srv1895164977=55, srv1130812493=4, srv757624712=88, srv1763814534=47, srv323490212=70, srv291424198=68, srv1406228504=23, srv1797740108=50, srv1260820477=15, srv1509429456=29, srv1736366041=44, srv1739853844=46, srv329010883=72, srv1434796053=26, srv218130788=64, srv1271158608=17, srv13268465=20, srv2125296306=61, srv2145265884=63, srv537141589=81, srv1398671422=22, srv2100236306=60, srv2140325979=62, srv91009691=97, srv661740573=84, srv1309690657=18, srv826618282=93, srv1518500164=30, srv825718935=92, srv598494785=83, srv1485851064=27, srv1143111949=5, srv74404835=87, srv244080472=65, srv400457983=74, srv763534872=89, srv1172115749=9, srv1711001418=40, srv828846443=94, srv721339763=86, srv785658370=90, srv1870592288=53, srv1353201827=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:16,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:16,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:16,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:16,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:16,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 04:53:16,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 04:53:16,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 04:53:16,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 04:53:16,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 04:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 04:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 04:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 04:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 04:53:16,784 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:16,784 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 04:53:16,784 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 04:53:17,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 540 ms to try 5000 different iterations.  Found a solution that moves 4727 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17518372477042823. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040034278864804915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05757540315325965, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5252222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040034278864804915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:17,185 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1730963918:82 , srv2125296306:82 , srv1165670430:83 , srv1353201827:83 , srv1737795890:83 , srv1763814534:83 , srv244080472:83 , srv537141589:83 , srv803952363:83 , srv1130812493:84 , srv1150685296:84 , srv1524512316:84 , srv186328237:84 , srv1870592288:84 , srv2145265884:84 , srv26920791:84 , srv661740573:84 , srv74404835:84 , srv757624712:84 , srv876605774:84 , srv1029595954:85 , srv1192029661:85 , srv1406228504:85 , srv1411547296:85 , srv1420310179:85 , srv1434796053:85 , srv1498951008:85 , srv1770863979:85 , srv1776247967:85 , srv1895164977:85 , srv1920606680:85 , srv291424198:85 , srv295199016:85 , srv469854768:85 , srv48201877:85 , srv707185847:85 , srv721339763:85 , srv1066914141:86 , srv1154402644:86 , srv119424825:86 , srv1242126737:86 , srv1326241449:86 , srv13268465:86 , srv156001983:86 , srv1613433912:86 , srv1736366041:86 , srv1963390247:86 , srv253774427:86 , srv32678487:86 , srv329010883:86 , srv44010105:86 , srv505596989:86 , srv558087435:86 , srv1191128800:87 , srv1260820477:87 , srv1309690657:87 , srv1398671422:87 , srv1518500164:87 , srv1711001418:87 , srv1721319151:87 , srv323490212:87 , srv400457983:87 , srv47271895:87 , srv474981157:87 , srv825718935:87 , srv826618282:87 , srv91009691:87 , srv946829752:87 , srv1130249989:88 , srv1143111949:88 , srv1172115749:88 , srv126976538:88 , srv1509429456:88 , srv1598472599:88 , srv1622732230:88 , srv1739853844:88 , srv1830858695:88 , srv1981864639:88 , srv218130788:88 , srv598494785:88 , srv763534872:88 , srv898687215:88 , srv1258580679:89 , srv1271158608:89 , srv1485851064:89 , srv1592761274:89 , srv1675767291:89 , srv1705281587:89 , srv1727460862:89 , srv1871747099:89 , srv785658370:89 , srv828846443:89 , srv958958202:89 , srv1633961469:90 , srv2013467195:90 , srv2100236306:90 , srv2140325979:91 , srv334624726:91 , srv101750058:96 , srv1797740108:446 }
2023-09-24 04:53:17,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:17,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv119424825=12, srv1737795890=45, srv1598472599=34, srv1830858695=51, srv469854768=76, srv186328237=52, srv958958202=99, srv1258580679=14, srv1192029661=11, srv47271895=77, srv1326241449=19, srv48201877=79, srv1730963918=43, srv1191128800=10, srv1242126737=13, srv946829752=98, srv505596989=80, srv876605774=95, srv558087435=82, srv1411547296=24, srv1920606680=56, srv1776247967=49, srv334624726=73, srv32678487=71, srv1150685296=6, srv1727460862=42, srv1524512316=31, srv898687215=96, srv1420310179=25, srv707185847=85, srv2013467195=59, srv295199016=69, srv1981864639=58, srv101750058=0, srv474981157=78, srv1871747099=54, srv126976538=16, srv1721319151=41, srv803952363=91, srv1770863979=48, srv1592761274=33, srv1029595954=1, srv156001983=32, srv1613433912=35, srv1622732230=36, srv1066914141=2, srv1963390247=57, srv253774427=66, srv1130249989=3, srv1165670430=8, srv1705281587=39, srv44010105=75, srv1633961469=37, srv26920791=67, srv1154402644=7, srv1675767291=38, srv1498951008=28, srv1895164977=55, srv1130812493=4, srv757624712=88, srv1763814534=47, srv323490212=70, srv291424198=68, srv1406228504=23, srv1797740108=50, srv1260820477=15, srv1509429456=29, srv1736366041=44, srv1739853844=46, srv329010883=72, srv1434796053=26, srv218130788=64, srv1271158608=17, srv13268465=20, srv2125296306=61, srv2145265884=63, srv537141589=81, srv1398671422=22, srv2100236306=60, srv2140325979=62, srv91009691=97, srv661740573=84, srv1309690657=18, srv826618282=93, srv1518500164=30, srv825718935=92, srv598494785=83, srv1485851064=27, srv1143111949=5, srv74404835=87, srv244080472=65, srv400457983=74, srv763534872=89, srv1172115749=9, srv1711001418=40, srv828846443=94, srv721339763=86, srv785658370=90, srv1870592288=53, srv1353201827=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:17,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:17,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:17,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:17,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:17,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:17,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 04:53:17,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 04:53:17,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 04:53:17,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 04:53:17,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 04:53:17,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 04:53:17,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 04:53:17,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 04:53:17,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 04:53:17,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 04:53:17,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 04:53:17,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 04:53:17,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 04:53:17,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 04:53:17,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 04:53:17,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 04:53:17,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 04:53:17,426 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040034278864804915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05757540315325965, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040034278864804915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:17,427 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 04:53:17,427 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17515062560898564, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040034278864804915, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05757540315325965, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040034278864804915, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23716666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 04:53:17,823 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 553 ms to try 5000 different iterations.  Found a solution that moves 2557 regions; Going from a computed imbalance of 0.17515062560898564 to a new imbalance of 0.013909540767091036. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204078569253787); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035832256659104667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2841111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204078569253787); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:17,837 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv101750058:77 , srv2140325979:80 , srv1271158608:81 , srv1258580679:82 , srv126976538:82 , srv1592761274:82 , srv1633961469:82 , srv1727460862:82 , srv1830858695:82 , srv1871747099:82 , srv2013467195:82 , srv334624726:82 , srv828846443:82 , srv958958202:82 , srv1130249989:83 , srv1143111949:83 , srv1598472599:83 , srv1981864639:83 , srv474981157:83 , srv598494785:83 , srv1066914141:84 , srv1242126737:84 , srv1260820477:84 , srv1309690657:84 , srv1326241449:84 , srv1398671422:84 , srv1485851064:84 , srv1509429456:84 , srv1518500164:84 , srv1675767291:84 , srv1705281587:84 , srv2100236306:84 , srv218130788:84 , srv785658370:84 , srv826618282:84 , srv91009691:84 , srv946829752:84 , srv1191128800:85 , srv119424825:85 , srv156001983:85 , srv1622732230:85 , srv1739853844:85 , srv329010883:85 , srv44010105:85 , srv558087435:85 , srv763534872:85 , srv898687215:85 , srv1154402644:86 , srv1172115749:86 , srv1192029661:86 , srv1420310179:86 , srv1524512316:86 , srv1711001418:86 , srv1963390247:86 , srv323490212:86 , srv47271895:86 , srv505596989:86 , srv707185847:86 , srv74404835:86 , srv825718935:86 , srv1029595954:87 , srv13268465:87 , srv1411547296:87 , srv1498951008:87 , srv1613433912:87 , srv1721319151:87 , srv1770863979:87 , srv2145265884:87 , srv291424198:87 , srv295199016:87 , srv400457983:87 , srv469854768:87 , srv48201877:87 , srv721339763:87 , srv1130812493:88 , srv1406228504:88 , srv1434796053:88 , srv1736366041:88 , srv1737795890:88 , srv186328237:88 , srv1895164977:88 , srv253774427:88 , srv32678487:88 , srv661740573:88 , srv876605774:88 , srv1150685296:89 , srv1353201827:89 , srv1730963918:89 , srv1763814534:89 , srv1920606680:89 , srv26920791:89 , srv757624712:89 , srv803952363:89 , srv1776247967:90 , srv2125296306:90 , srv244080472:90 , srv1870592288:91 , srv537141589:91 , srv1165670430:92 , srv1797740108:524 }
2023-09-24 04:53:17,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:17,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv119424825=12, srv1737795890=45, srv1598472599=34, srv1830858695=51, srv469854768=76, srv186328237=52, srv958958202=99, srv1258580679=14, srv1192029661=11, srv47271895=77, srv1326241449=19, srv48201877=79, srv1730963918=43, srv1191128800=10, srv1242126737=13, srv946829752=98, srv505596989=80, srv876605774=95, srv558087435=82, srv1411547296=24, srv1920606680=56, srv1776247967=49, srv334624726=73, srv32678487=71, srv1150685296=6, srv1727460862=42, srv1524512316=31, srv898687215=96, srv1420310179=25, srv707185847=85, srv2013467195=59, srv295199016=69, srv1981864639=58, srv101750058=0, srv474981157=78, srv1871747099=54, srv126976538=16, srv1721319151=41, srv803952363=91, srv1770863979=48, srv1592761274=33, srv1029595954=1, srv156001983=32, srv1613433912=35, srv1622732230=36, srv1066914141=2, srv1963390247=57, srv253774427=66, srv1130249989=3, srv1165670430=8, srv1705281587=39, srv44010105=75, srv1633961469=37, srv26920791=67, srv1154402644=7, srv1675767291=38, srv1498951008=28, srv1895164977=55, srv1130812493=4, srv757624712=88, srv1763814534=47, srv323490212=70, srv291424198=68, srv1406228504=23, srv1797740108=50, srv1260820477=15, srv1509429456=29, srv1736366041=44, srv1739853844=46, srv329010883=72, srv1434796053=26, srv218130788=64, srv1271158608=17, srv13268465=20, srv2125296306=61, srv2145265884=63, srv537141589=81, srv1398671422=22, srv2100236306=60, srv2140325979=62, srv91009691=97, srv661740573=84, srv1309690657=18, srv826618282=93, srv1518500164=30, srv825718935=92, srv598494785=83, srv1485851064=27, srv1143111949=5, srv74404835=87, srv244080472=65, srv400457983=74, srv763534872=89, srv1172115749=9, srv1711001418=40, srv828846443=94, srv721339763=86, srv785658370=90, srv1870592288=53, srv1353201827=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:17,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:17,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:17,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:17,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:17,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 04:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 04:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 04:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 04:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 04:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 04:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 04:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 04:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 04:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 04:53:17,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 04:53:17,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 04:53:17,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 04:53:18,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204078569253787); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035832256659104667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204078569253787); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:18,054 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 04:53:18,055 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013891636270410557, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02204078569253787); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035832256659104667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02204078569253787); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 04:53:18,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 529 ms to try 5000 different iterations.  Found a solution that moves 1039 regions; Going from a computed imbalance of 0.013891636270410557 to a new imbalance of 4.3375909618802497E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005320504585893126); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.11544444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005320504585893126); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:18,440 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1165670430:82 , srv1776247967:82 , srv1172115749:83 , srv1721319151:83 , srv1870592288:83 , srv1920606680:83 , srv2100236306:83 , srv253774427:83 , srv32678487:83 , srv101750058:84 , srv1150685296:84 , srv1434796053:84 , srv1622732230:84 , srv1705281587:84 , srv1736366041:84 , srv1739853844:84 , srv323490212:84 , srv400457983:84 , srv537141589:84 , srv763534872:84 , srv825718935:84 , srv898687215:84 , srv1130812493:85 , srv1154402644:85 , srv1191128800:85 , srv13268465:85 , srv1406228504:85 , srv1485851064:85 , srv1498951008:85 , srv1613433912:85 , srv1675767291:85 , srv1711001418:85 , srv1770863979:85 , srv186328237:85 , srv1895164977:85 , srv1963390247:85 , srv1981864639:85 , srv2013467195:85 , srv244080472:85 , srv26920791:85 , srv295199016:85 , srv47271895:85 , srv48201877:85 , srv721339763:85 , srv757624712:85 , srv785658370:85 , srv1029595954:86 , srv1066914141:86 , srv1192029661:86 , srv119424825:86 , srv1309690657:86 , srv1353201827:86 , srv1398671422:86 , srv1411547296:86 , srv1420310179:86 , srv1518500164:86 , srv156001983:86 , srv1598472599:86 , srv1727460862:86 , srv1730963918:86 , srv1737795890:86 , srv1763814534:86 , srv1871747099:86 , srv2125296306:86 , srv218130788:86 , srv291424198:86 , srv329010883:86 , srv334624726:86 , srv44010105:86 , srv469854768:86 , srv558087435:86 , srv598494785:86 , srv661740573:86 , srv803952363:86 , srv876605774:86 , srv946829752:86 , srv1130249989:87 , srv1143111949:87 , srv1258580679:87 , srv1260820477:87 , srv126976538:87 , srv1509429456:87 , srv1592761274:87 , srv1633961469:87 , srv2140325979:87 , srv2145265884:87 , srv474981157:87 , srv505596989:87 , srv707185847:87 , srv826618282:87 , srv828846443:87 , srv91009691:87 , srv1242126737:88 , srv1271158608:88 , srv1326241449:88 , srv1524512316:88 , srv1830858695:88 , srv74404835:88 , srv958958202:88 , srv1797740108:535 }
2023-09-24 04:53:18,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:18,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv119424825=12, srv1737795890=45, srv1598472599=34, srv1830858695=51, srv469854768=76, srv186328237=52, srv958958202=99, srv1258580679=14, srv1192029661=11, srv47271895=77, srv1326241449=19, srv48201877=79, srv1730963918=43, srv1191128800=10, srv1242126737=13, srv946829752=98, srv505596989=80, srv876605774=95, srv558087435=82, srv1411547296=24, srv1920606680=56, srv1776247967=49, srv334624726=73, srv32678487=71, srv1150685296=6, srv1727460862=42, srv1524512316=31, srv898687215=96, srv1420310179=25, srv707185847=85, srv2013467195=59, srv295199016=69, srv1981864639=58, srv101750058=0, srv474981157=78, srv1871747099=54, srv126976538=16, srv1721319151=41, srv803952363=91, srv1770863979=48, srv1592761274=33, srv1029595954=1, srv156001983=32, srv1613433912=35, srv1622732230=36, srv1066914141=2, srv1963390247=57, srv253774427=66, srv1130249989=3, srv1165670430=8, srv1705281587=39, srv44010105=75, srv1633961469=37, srv26920791=67, srv1154402644=7, srv1675767291=38, srv1498951008=28, srv1895164977=55, srv1130812493=4, srv757624712=88, srv1763814534=47, srv323490212=70, srv291424198=68, srv1406228504=23, srv1797740108=50, srv1260820477=15, srv1509429456=29, srv1736366041=44, srv1739853844=46, srv329010883=72, srv1434796053=26, srv218130788=64, srv1271158608=17, srv13268465=20, srv2125296306=61, srv2145265884=63, srv537141589=81, srv1398671422=22, srv2100236306=60, srv2140325979=62, srv91009691=97, srv661740573=84, srv1309690657=18, srv826618282=93, srv1518500164=30, srv825718935=92, srv598494785=83, srv1485851064=27, srv1143111949=5, srv74404835=87, srv244080472=65, srv400457983=74, srv763534872=89, srv1172115749=9, srv1711001418=40, srv828846443=94, srv721339763=86, srv785658370=90, srv1870592288=53, srv1353201827=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:18,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:18,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:18,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:18,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:18,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:18,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:18,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:18,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:18,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:18,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:18,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:18,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:18,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:18,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:18,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:18,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 04:53:18,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 04:53:18,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 04:53:18,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 04:53:18,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 04:53:18,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 04:53:18,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 04:53:18,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 04:53:18,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 04:53:18,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 04:53:18,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 04:53:18,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 04:53:18,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 04:53:18,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 04:53:18,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 04:53:18,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 04:53:18,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 04:53:18,646 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005320504585893126); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005320504585893126); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:18,646 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 04:53:18,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.2648386269134145E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005320504585893126); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005320504585893126); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 04:53:18,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 452 ms to try 5000 different iterations.  Found a solution that moves 218 regions; Going from a computed imbalance of 4.2648386269134145E-4 to a new imbalance of 2.022833135971505E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.02422222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:18,962 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1509429456:83 , srv1592761274:83 , srv1633961469:83 , srv2140325979:83 , srv291424198:83 , srv334624726:83 , srv505596989:83 , srv828846443:83 , srv958958202:83 , srv1029595954:84 , srv1143111949:84 , srv1242126737:84 , srv1258580679:84 , srv1260820477:84 , srv1271158608:84 , srv1326241449:84 , srv13268465:84 , srv1353201827:84 , srv1406228504:84 , srv1411547296:84 , srv1524512316:84 , srv1675767291:84 , srv1711001418:84 , srv1727460862:84 , srv1736366041:84 , srv1830858695:84 , srv1870592288:84 , srv1895164977:84 , srv2125296306:84 , srv2145265884:84 , srv218130788:84 , srv400457983:84 , srv469854768:84 , srv47271895:84 , srv474981157:84 , srv537141589:84 , srv598494785:84 , srv661740573:84 , srv707185847:84 , srv74404835:84 , srv757624712:84 , srv785658370:84 , srv803952363:84 , srv826618282:84 , srv876605774:84 , srv91009691:84 , srv101750058:85 , srv1066914141:85 , srv1130249989:85 , srv1130812493:85 , srv1150685296:85 , srv1154402644:85 , srv1165670430:85 , srv1172115749:85 , srv1191128800:85 , srv1192029661:85 , srv119424825:85 , srv126976538:85 , srv1309690657:85 , srv1398671422:85 , srv1420310179:85 , srv1434796053:85 , srv1485851064:85 , srv1498951008:85 , srv1518500164:85 , srv156001983:85 , srv1598472599:85 , srv1613433912:85 , srv1622732230:85 , srv1705281587:85 , srv1721319151:85 , srv1730963918:85 , srv1737795890:85 , srv1739853844:85 , srv1763814534:85 , srv1770863979:85 , srv1776247967:85 , srv186328237:85 , srv1871747099:85 , srv1920606680:85 , srv1963390247:85 , srv2013467195:85 , srv2100236306:85 , srv244080472:85 , srv26920791:85 , srv323490212:85 , srv32678487:85 , srv329010883:85 , srv44010105:85 , srv48201877:85 , srv558087435:85 , srv721339763:85 , srv763534872:85 , srv825718935:85 , srv898687215:85 , srv946829752:85 , srv1981864639:86 , srv253774427:86 , srv295199016:86 , srv1797740108:637 }
2023-09-24 04:53:18,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:18,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv119424825=12, srv1737795890=45, srv1598472599=34, srv1830858695=51, srv469854768=76, srv186328237=52, srv958958202=99, srv1258580679=14, srv1192029661=11, srv47271895=77, srv1326241449=19, srv48201877=79, srv1730963918=43, srv1191128800=10, srv1242126737=13, srv946829752=98, srv505596989=80, srv876605774=95, srv558087435=82, srv1411547296=24, srv1920606680=56, srv1776247967=49, srv334624726=73, srv32678487=71, srv1150685296=6, srv1727460862=42, srv1524512316=31, srv898687215=96, srv1420310179=25, srv707185847=85, srv2013467195=59, srv295199016=69, srv1981864639=58, srv101750058=0, srv474981157=78, srv1871747099=54, srv126976538=16, srv1721319151=41, srv803952363=91, srv1770863979=48, srv1592761274=33, srv1029595954=1, srv156001983=32, srv1613433912=35, srv1622732230=36, srv1066914141=2, srv1963390247=57, srv253774427=66, srv1130249989=3, srv1165670430=8, srv1705281587=39, srv44010105=75, srv1633961469=37, srv26920791=67, srv1154402644=7, srv1675767291=38, srv1498951008=28, srv1895164977=55, srv1130812493=4, srv757624712=88, srv1763814534=47, srv323490212=70, srv291424198=68, srv1406228504=23, srv1797740108=50, srv1260820477=15, srv1509429456=29, srv1736366041=44, srv1739853844=46, srv329010883=72, srv1434796053=26, srv218130788=64, srv1271158608=17, srv13268465=20, srv2125296306=61, srv2145265884=63, srv537141589=81, srv1398671422=22, srv2100236306=60, srv2140325979=62, srv91009691=97, srv661740573=84, srv1309690657=18, srv826618282=93, srv1518500164=30, srv825718935=92, srv598494785=83, srv1485851064=27, srv1143111949=5, srv74404835=87, srv244080472=65, srv400457983=74, srv763534872=89, srv1172115749=9, srv1711001418=40, srv828846443=94, srv721339763=86, srv785658370=90, srv1870592288=53, srv1353201827=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:18,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 04:53:18,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 04:53:18,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 04:53:18,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 04:53:18,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 04:53:18,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 04:53:18,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 04:53:18,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 04:53:19,087 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8701862733846913E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:19,088 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1509429456:83 , srv1592761274:83 , srv1633961469:83 , srv2140325979:83 , srv291424198:83 , srv334624726:83 , srv505596989:83 , srv828846443:83 , srv958958202:83 , srv1029595954:84 , srv1143111949:84 , srv1242126737:84 , srv1258580679:84 , srv1260820477:84 , srv1271158608:84 , srv1326241449:84 , srv13268465:84 , srv1353201827:84 , srv1406228504:84 , srv1411547296:84 , srv1524512316:84 , srv1675767291:84 , srv1711001418:84 , srv1727460862:84 , srv1736366041:84 , srv1830858695:84 , srv1870592288:84 , srv1895164977:84 , srv2125296306:84 , srv2145265884:84 , srv218130788:84 , srv400457983:84 , srv469854768:84 , srv47271895:84 , srv474981157:84 , srv537141589:84 , srv598494785:84 , srv661740573:84 , srv707185847:84 , srv74404835:84 , srv757624712:84 , srv785658370:84 , srv803952363:84 , srv826618282:84 , srv876605774:84 , srv91009691:84 , srv101750058:85 , srv1066914141:85 , srv1130249989:85 , srv1130812493:85 , srv1150685296:85 , srv1154402644:85 , srv1165670430:85 , srv1172115749:85 , srv1191128800:85 , srv1192029661:85 , srv119424825:85 , srv126976538:85 , srv1309690657:85 , srv1398671422:85 , srv1420310179:85 , srv1434796053:85 , srv1485851064:85 , srv1498951008:85 , srv1518500164:85 , srv156001983:85 , srv1598472599:85 , srv1613433912:85 , srv1622732230:85 , srv1705281587:85 , srv1721319151:85 , srv1730963918:85 , srv1737795890:85 , srv1739853844:85 , srv1763814534:85 , srv1770863979:85 , srv1776247967:85 , srv186328237:85 , srv1871747099:85 , srv1920606680:85 , srv1963390247:85 , srv2013467195:85 , srv2100236306:85 , srv244080472:85 , srv26920791:85 , srv323490212:85 , srv32678487:85 , srv329010883:85 , srv44010105:85 , srv48201877:85 , srv558087435:85 , srv721339763:85 , srv763534872:85 , srv825718935:85 , srv898687215:85 , srv946829752:85 , srv1981864639:86 , srv253774427:86 , srv295199016:86 , srv1797740108:637 }
2023-09-24 04:53:19,115 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=390 (was 364) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 173), AvailableMemoryMB=9465 (was 10080)
2023-09-24 04:53:19,122 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=390, ProcessCount=171, AvailableMemoryMB=9465
2023-09-24 04:53:19,123 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 04:53:19,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-24 04:53:19,131 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004549696:84 , srv1019463491:84 , srv1275298154:84 , srv1304133845:84 , srv1369539823:84 , srv1401179213:84 , srv1403089039:84 , srv1436173963:84 , srv151253258:84 , srv1610690489:84 , srv1797318568:84 , srv1813454206:84 , srv1863226146:84 , srv1863865932:84 , srv1878822955:84 , srv1900856174:84 , srv1914155674:84 , srv1927472862:84 , srv1979490080:84 , srv2021509057:84 , srv2031879018:84 , srv2134119844:84 , srv347804754:84 , srv435609123:84 , srv507388111:84 , srv60314686:84 , srv620489424:84 , srv68596424:84 , srv731980742:84 , srv866799738:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-24 04:53:19,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:53:19,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2021509057=19, srv435609123=23, srv1436173963=7, srv1610690489=9, srv1914155674=16, srv1797318568=10, srv2134119844=21, srv1900856174=15, srv1927472862=17, srv1019463491=1, srv347804754=22, srv68596424=27, srv1004549696=0, srv866799738=29, srv1813454206=11, srv1401179213=5, srv151253258=8, srv1403089039=6, srv1863226146=12, srv507388111=24, srv1878822955=14, srv731980742=28, srv1369539823=4, srv1304133845=3, srv60314686=25, srv1979490080=18, srv1863865932=13, srv1275298154=2, srv2031879018=20, srv620489424=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:53:19,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:53:19,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:53:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:53:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:53:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:53:19,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-24 04:53:19,160 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1347306307360752, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:53:19,160 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912617575082369, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1347306307360752, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-24 04:55:19,146 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5549830 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912617575082369 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:55:19,153 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004549696:90 , srv1019463491:90 , srv1275298154:90 , srv1304133845:90 , srv1369539823:90 , srv1401179213:90 , srv1403089039:90 , srv1436173963:90 , srv151253258:90 , srv1610690489:90 , srv1797318568:90 , srv1813454206:90 , srv1863226146:90 , srv1863865932:90 , srv1878822955:90 , srv1900856174:90 , srv1914155674:90 , srv1927472862:90 , srv1979490080:90 , srv2021509057:90 , srv2031879018:90 , srv2134119844:90 , srv347804754:90 , srv435609123:90 , srv507388111:90 , srv60314686:90 , srv620489424:90 , srv68596424:90 , srv731980742:90 , srv866799738:90 }
2023-09-24 04:55:19,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 04:55:19,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2021509057=19, srv435609123=23, srv1436173963=7, srv1610690489=9, srv1914155674=16, srv1797318568=10, srv2134119844=21, srv1900856174=15, srv1927472862=17, srv1019463491=1, srv347804754=22, srv68596424=27, srv1004549696=0, srv866799738=29, srv1813454206=11, srv1401179213=5, srv151253258=8, srv1403089039=6, srv1863226146=12, srv507388111=24, srv1878822955=14, srv731980742=28, srv1369539823=4, srv1304133845=3, srv60314686=25, srv1979490080=18, srv1863865932=13, srv1275298154=2, srv2031879018=20, srv620489424=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 04:55:19,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 04:55:19,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 04:55:19,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 04:55:19,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 04:55:19,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-24 04:55:19,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 04:55:19,171 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004549696:90 , srv1019463491:90 , srv1275298154:90 , srv1304133845:90 , srv1369539823:90 , srv1401179213:90 , srv1403089039:90 , srv1436173963:90 , srv151253258:90 , srv1610690489:90 , srv1797318568:90 , srv1813454206:90 , srv1863226146:90 , srv1863865932:90 , srv1878822955:90 , srv1900856174:90 , srv1914155674:90 , srv1927472862:90 , srv1979490080:90 , srv2021509057:90 , srv2031879018:90 , srv2134119844:90 , srv347804754:90 , srv435609123:90 , srv507388111:90 , srv60314686:90 , srv620489424:90 , srv68596424:90 , srv731980742:90 , srv866799738:90 }
2023-09-24 04:55:19,183 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 390), ProcessCount=170 (was 171), AvailableMemoryMB=8427 (was 9465)