2024-04-10 00:53:20,274 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-10 00:53:20,603 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-10 00:53:20,720 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-10 00:53:20,789 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 00:53:20,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-10 00:53:20,812 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=360, ProcessCount=169, AvailableMemoryMB=16477
2024-04-10 00:53:20,814 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 00:53:20,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-10 00:53:20,952 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1037156773:84 , srv10444358:84 , srv1075414181:84 , srv1102250671:84 , srv1119148680:84 , srv1164108141:84 , srv1174493252:84 , srv1178371048:84 , srv1182984431:84 , srv1188989733:84 , srv1190940957:84 , srv1191985913:84 , srv1250813762:84 , srv1262374980:84 , srv1273773142:84 , srv1279617782:84 , srv1293852394:84 , srv1308153942:84 , srv1321974888:84 , srv1343054369:84 , srv1387106497:84 , srv1395633367:84 , srv1400878673:84 , srv1407056356:84 , srv1408499621:84 , srv1440571341:84 , srv14593139:84 , srv1505916535:84 , srv1506698678:84 , srv1587823610:84 , srv1608605212:84 , srv1661768674:84 , srv1672893819:84 , srv1689569626:84 , srv1693533622:84 , srv1712966335:84 , srv1715664882:84 , srv1751404572:84 , srv1767562723:84 , srv1779040107:84 , srv1792802097:84 , srv1803455787:84 , srv1863701352:84 , srv188169842:84 , srv1888199607:84 , srv191412149:84 , srv193339352:84 , srv1934195587:84 , srv1937549354:84 , srv1940190747:84 , srv1950114482:84 , srv2007181186:84 , srv2060078515:84 , srv2066283342:84 , srv211471933:84 , srv2131966266:84 , srv2141192270:84 , srv2147398586:84 , srv230861697:84 , srv242708308:84 , srv243430031:84 , srv244054924:84 , srv256012864:84 , srv256642:84 , srv263414787:84 , srv277115918:84 , srv32756278:84 , srv333406664:84 , srv340293031:84 , srv363710411:84 , srv387823670:84 , srv392854594:84 , srv481441539:84 , srv485453467:84 , srv522929612:84 , srv528026438:84 , srv529231342:84 , srv591505995:84 , srv594841708:84 , srv598409157:84 , srv603767746:84 , srv612870911:84 , srv634733402:84 , srv637009635:84 , srv638052247:84 , srv653433066:84 , srv665735732:84 , srv699715809:84 , srv709759951:84 , srv736463011:84 , srv745027173:84 , srv811948993:84 , srv815823814:84 , srv822832817:84 , srv857142480:84 , srv885435734:84 , srv894588696:84 , srv901256406:84 , srv932219680:84 , srv1731913698:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-10 00:53:20,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:20,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:21,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:21,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:21,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:21,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:21,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:21,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:21,072 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:21,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:21,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:21,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:21,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:21,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:21,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:21,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:21,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:21,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:21,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:21,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:21,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:21,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:21,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:21,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:21,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:21,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:21,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:21,457 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:21,457 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 00:53:21,458 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 00:53:21,925 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 804 ms to try 5000 different iterations.  Found a solution that moves 4643 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18485499779352127. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041167643964522965, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05500535633544451, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5158888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041167643964522965, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17941666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25383333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:21,940 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1395633367:81 , srv1037156773:82 , srv1279617782:82 , srv1661768674:82 , srv1715664882:82 , srv256642:82 , srv811948993:82 , srv1321974888:83 , srv1387106497:83 , srv1506698678:83 , srv1767562723:83 , srv1792802097:83 , srv1407056356:84 , srv1440571341:84 , srv1751404572:84 , srv1779040107:84 , srv1803455787:84 , srv1940190747:84 , srv277115918:84 , srv529231342:84 , srv634733402:84 , srv709759951:84 , srv901256406:84 , srv932219680:84 , srv1182984431:85 , srv1672893819:85 , srv1712966335:85 , srv1950114482:85 , srv211471933:85 , srv2141192270:85 , srv263414787:85 , srv387823670:85 , srv594841708:85 , srv745027173:85 , srv815823814:85 , srv822832817:85 , srv1102250671:86 , srv1164108141:86 , srv1174493252:86 , srv1178371048:86 , srv1191985913:86 , srv1308153942:86 , srv14593139:86 , srv1608605212:86 , srv1888199607:86 , srv1934195587:86 , srv2066283342:86 , srv256012864:86 , srv528026438:86 , srv665735732:86 , srv699715809:86 , srv885435734:86 , srv894588696:86 , srv10444358:87 , srv1188989733:87 , srv1262374980:87 , srv1343054369:87 , srv1505916535:87 , srv1693533622:87 , srv1863701352:87 , srv188169842:87 , srv191412149:87 , srv193339352:87 , srv2007181186:87 , srv2060078515:87 , srv2131966266:87 , srv230861697:87 , srv244054924:87 , srv32756278:87 , srv333406664:87 , srv363710411:87 , srv603767746:87 , srv1119148680:88 , srv1190940957:88 , srv1273773142:88 , srv1689569626:88 , srv340293031:88 , srv485453467:88 , srv591505995:88 , srv653433066:88 , srv857142480:88 , srv1075414181:89 , srv1250813762:89 , srv1400878673:89 , srv392854594:89 , srv522929612:89 , srv612870911:89 , srv637009635:89 , srv1293852394:90 , srv1937549354:90 , srv2147398586:90 , srv242708308:90 , srv243430031:90 , srv598409157:90 , srv638052247:90 , srv1408499621:91 , srv1587823610:91 , srv481441539:93 , srv736463011:93 , srv1731913698:456 }
2024-04-10 00:53:21,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:21,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:21,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:21,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:21,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:21,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:21,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:21,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:21,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:21,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:21,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:21,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:21,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:21,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:21,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:21,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:21,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:22,153 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041167643964522965, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05500535633544451, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041167643964522965, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17941666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25383333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:22,154 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 00:53:22,154 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18482248681265015, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041167643964522965, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05500535633544451, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041167643964522965, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17941666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.25383333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 00:53:22,448 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 436 ms to try 5000 different iterations.  Found a solution that moves 2699 regions; Going from a computed imbalance of 0.18482248681265015 to a new imbalance of 0.01622381357536144. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0214971497485744); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03531801967875861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29988888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0214971497485744); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04791666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:22,461 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1587823610:79 , srv481441539:80 , srv736463011:80 , srv1937549354:81 , srv2147398586:81 , srv1250813762:82 , srv1293852394:82 , srv1408499621:82 , srv243430031:82 , srv591505995:82 , srv612870911:82 , srv1075414181:83 , srv1308153942:83 , srv1400878673:83 , srv242708308:83 , srv485453467:83 , srv522929612:83 , srv598409157:83 , srv637009635:83 , srv653433066:83 , srv857142480:83 , srv10444358:84 , srv1188989733:84 , srv1190940957:84 , srv1273773142:84 , srv1689569626:84 , srv2066283342:84 , srv230861697:84 , srv340293031:84 , srv603767746:84 , srv665735732:84 , srv1119148680:85 , srv1164108141:85 , srv1178371048:85 , srv1608605212:85 , srv1693533622:85 , srv191412149:85 , srv1934195587:85 , srv333406664:85 , srv363710411:85 , srv387823670:85 , srv392854594:85 , srv594841708:85 , srv638052247:85 , srv815823814:85 , srv885435734:85 , srv894588696:85 , srv1174493252:86 , srv1191985913:86 , srv1343054369:86 , srv1505916535:86 , srv1672893819:86 , srv1779040107:86 , srv1803455787:86 , srv2060078515:86 , srv244054924:86 , srv256012864:86 , srv709759951:86 , srv1102250671:87 , srv1407056356:87 , srv1440571341:87 , srv14593139:87 , srv1661768674:87 , srv1712966335:87 , srv1751404572:87 , srv1863701352:87 , srv188169842:87 , srv1888199607:87 , srv2007181186:87 , srv263414787:87 , srv528026438:87 , srv529231342:87 , srv699715809:87 , srv822832817:87 , srv1182984431:88 , srv1262374980:88 , srv1767562723:88 , srv211471933:88 , srv2131966266:88 , srv2141192270:88 , srv32756278:88 , srv634733402:88 , srv811948993:88 , srv1321974888:89 , srv1387106497:89 , srv1792802097:89 , srv193339352:89 , srv277115918:89 , srv745027173:89 , srv932219680:89 , srv1037156773:90 , srv1279617782:90 , srv1395633367:90 , srv1506698678:90 , srv1940190747:90 , srv1950114482:90 , srv901256406:90 , srv1715664882:92 , srv256642:93 , srv1731913698:509 }
2024-04-10 00:53:22,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:22,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:22,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:22,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:22,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:22,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:22,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:22,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:22,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:22,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:22,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:22,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:22,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:22,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:22,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:22,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:22,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:22,684 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0214971497485744); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03531801967875861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0214971497485744); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04791666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:22,684 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 00:53:22,685 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01620491477342924, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0214971497485744); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03531801967875861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0214971497485744); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04791666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 00:53:23,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 480 ms to try 5000 different iterations.  Found a solution that moves 1092 regions; Going from a computed imbalance of 0.01620491477342924 to a new imbalance of 5.586253478459858E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006785326333203205); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008475223015516377); MoveCostFunction : (multiplier=7.0, imbalance=0.12133333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006785326333203205); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:23,031 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1262374980:81 , srv193339352:81 , srv2131966266:82 , srv256642:82 , srv32756278:82 , srv1715664882:83 , srv1863701352:83 , srv1888199607:83 , srv1950114482:83 , srv638052247:83 , srv901256406:83 , srv1102250671:84 , srv1119148680:84 , srv1182984431:84 , srv1190940957:84 , srv1343054369:84 , srv1505916535:84 , srv1506698678:84 , srv188169842:84 , srv1940190747:84 , srv2007181186:84 , srv2060078515:84 , srv2141192270:84 , srv242708308:84 , srv244054924:84 , srv277115918:84 , srv481441539:84 , srv699715809:84 , srv736463011:84 , srv745027173:84 , srv932219680:84 , srv1321974888:85 , srv1387106497:85 , srv1407056356:85 , srv1408499621:85 , srv14593139:85 , srv1608605212:85 , srv1689569626:85 , srv1693533622:85 , srv1712966335:85 , srv1792802097:85 , srv211471933:85 , srv243430031:85 , srv256012864:85 , srv392854594:85 , srv522929612:85 , srv598409157:85 , srv634733402:85 , srv637009635:85 , srv894588696:85 , srv1075414181:86 , srv1164108141:86 , srv1174493252:86 , srv1191985913:86 , srv1273773142:86 , srv1279617782:86 , srv1293852394:86 , srv1400878673:86 , srv1440571341:86 , srv1672893819:86 , srv191412149:86 , srv1937549354:86 , srv2147398586:86 , srv230861697:86 , srv263414787:86 , srv333406664:86 , srv340293031:86 , srv363710411:86 , srv485453467:86 , srv528026438:86 , srv529231342:86 , srv612870911:86 , srv665735732:86 , srv811948993:86 , srv822832817:86 , srv857142480:86 , srv1037156773:87 , srv10444358:87 , srv1250813762:87 , srv1395633367:87 , srv1587823610:87 , srv1751404572:87 , srv1767562723:87 , srv1803455787:87 , srv1934195587:87 , srv591505995:87 , srv594841708:87 , srv603767746:87 , srv653433066:87 , srv709759951:87 , srv815823814:87 , srv885435734:87 , srv1188989733:88 , srv1308153942:88 , srv1661768674:88 , srv1779040107:88 , srv2066283342:88 , srv1178371048:89 , srv387823670:89 , srv1731913698:553 }
2024-04-10 00:53:23,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:23,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:23,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:23,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:23,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:23,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:23,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:23,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:23,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:23,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:23,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:23,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:23,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:23,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:23,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:23,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:23,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:23,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:23,219 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006785326333203205); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008475223015516377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006785326333203205); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:23,219 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 00:53:23,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.509790004173253E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006785326333203205); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008475223015516377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006785326333203205); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 00:53:23,617 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 533 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 5.509790004173253E-4 to a new imbalance of 2.7689543530320244E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:23,618 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1178371048:82 , srv1037156773:83 , srv1188989733:83 , srv211471933:83 , srv387823670:83 , srv392854594:83 , srv528026438:83 , srv603767746:83 , srv885435734:83 , srv10444358:84 , srv1075414181:84 , srv1174493252:84 , srv1191985913:84 , srv1250813762:84 , srv1273773142:84 , srv1279617782:84 , srv1293852394:84 , srv1395633367:84 , srv1400878673:84 , srv1408499621:84 , srv1440571341:84 , srv14593139:84 , srv1587823610:84 , srv1751404572:84 , srv1767562723:84 , srv188169842:84 , srv193339352:84 , srv1934195587:84 , srv1940190747:84 , srv1950114482:84 , srv2007181186:84 , srv2066283342:84 , srv243430031:84 , srv263414787:84 , srv277115918:84 , srv340293031:84 , srv591505995:84 , srv598409157:84 , srv638052247:84 , srv653433066:84 , srv745027173:84 , srv822832817:84 , srv1102250671:85 , srv1119148680:85 , srv1164108141:85 , srv1182984431:85 , srv1262374980:85 , srv1308153942:85 , srv1321974888:85 , srv1343054369:85 , srv1387106497:85 , srv1505916535:85 , srv1506698678:85 , srv1672893819:85 , srv1689569626:85 , srv1712966335:85 , srv1715664882:85 , srv1779040107:85 , srv1792802097:85 , srv1803455787:85 , srv1863701352:85 , srv191412149:85 , srv1937549354:85 , srv2060078515:85 , srv2131966266:85 , srv2141192270:85 , srv2147398586:85 , srv230861697:85 , srv242708308:85 , srv244054924:85 , srv256012864:85 , srv256642:85 , srv32756278:85 , srv333406664:85 , srv363710411:85 , srv481441539:85 , srv485453467:85 , srv529231342:85 , srv594841708:85 , srv612870911:85 , srv634733402:85 , srv637009635:85 , srv699715809:85 , srv709759951:85 , srv815823814:85 , srv894588696:85 , srv901256406:85 , srv932219680:85 , srv1190940957:86 , srv1407056356:86 , srv1661768674:86 , srv1693533622:86 , srv1888199607:86 , srv522929612:86 , srv665735732:86 , srv736463011:86 , srv811948993:86 , srv857142480:86 , srv1608605212:87 , srv1731913698:625 }
2024-04-10 00:53:23,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:23,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:23,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:23,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:23,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:23,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:23,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:23,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:23,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:23,746 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:23,746 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-10 00:53:23,746 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6058042659552916E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-10 00:53:23,993 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 345 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 2.6058042659552916E-5 to a new imbalance of 6.912120645670642E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:23,993 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1608605212:83 , srv1661768674:83 , srv1693533622:83 , srv1779040107:83 , srv191412149:83 , srv333406664:83 , srv363710411:83 , srv522929612:83 , srv736463011:83 , srv857142480:83 , srv1037156773:84 , srv10444358:84 , srv1075414181:84 , srv1102250671:84 , srv1119148680:84 , srv1164108141:84 , srv1174493252:84 , srv1178371048:84 , srv1182984431:84 , srv1188989733:84 , srv1190940957:84 , srv1191985913:84 , srv1250813762:84 , srv1273773142:84 , srv1279617782:84 , srv1293852394:84 , srv1308153942:84 , srv1321974888:84 , srv1343054369:84 , srv1387106497:84 , srv1395633367:84 , srv1400878673:84 , srv1407056356:84 , srv1408499621:84 , srv14593139:84 , srv1505916535:84 , srv1506698678:84 , srv1672893819:84 , srv1689569626:84 , srv1712966335:84 , srv1715664882:84 , srv1751404572:84 , srv1767562723:84 , srv1792802097:84 , srv1803455787:84 , srv1863701352:84 , srv188169842:84 , srv1888199607:84 , srv1934195587:84 , srv1937549354:84 , srv1940190747:84 , srv1950114482:84 , srv2007181186:84 , srv2060078515:84 , srv2066283342:84 , srv2131966266:84 , srv2141192270:84 , srv2147398586:84 , srv230861697:84 , srv242708308:84 , srv244054924:84 , srv256012864:84 , srv256642:84 , srv263414787:84 , srv32756278:84 , srv340293031:84 , srv387823670:84 , srv392854594:84 , srv481441539:84 , srv485453467:84 , srv528026438:84 , srv529231342:84 , srv594841708:84 , srv598409157:84 , srv612870911:84 , srv634733402:84 , srv637009635:84 , srv638052247:84 , srv653433066:84 , srv665735732:84 , srv699715809:84 , srv709759951:84 , srv745027173:84 , srv811948993:84 , srv815823814:84 , srv822832817:84 , srv901256406:84 , srv932219680:84 , srv1262374980:85 , srv1440571341:85 , srv1587823610:85 , srv193339352:85 , srv211471933:85 , srv243430031:85 , srv277115918:85 , srv591505995:85 , srv603767746:85 , srv885435734:85 , srv894588696:85 , srv1731913698:683 }
2024-04-10 00:53:23,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:23,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1792802097=41, srv1037156773=0, srv1343054369=19, srv2007181186=52, srv481441539=73, srv2141192270=57, srv1174493252=6, srv387823670=71, srv653433066=86, srv263414787=65, srv822832817=94, srv591505995=78, srv638052247=85, srv665735732=87, srv333406664=68, srv815823814=93, srv529231342=77, srv211471933=55, srv637009635=84, srv485453467=74, srv1779040107=40, srv1075414181=2, srv1387106497=20, srv1731913698=37, srv363710411=70, srv1689569626=33, srv901256406=98, srv1407056356=23, srv230861697=59, srv1190940957=10, srv1395633367=21, srv32756278=67, srv1164108141=5, srv1506698678=28, srv612870911=82, srv745027173=91, srv894588696=97, srv14593139=26, srv2066283342=54, srv10444358=1, srv340293031=69, srv528026438=76, srv522929612=75, srv1693533622=34, srv603767746=81, srv2060078515=53, srv1400878673=22, srv1440571341=25, srv1767562723=39, srv191412149=46, srv1408499621=24, srv1188989733=9, srv1587823610=29, srv811948993=92, srv736463011=90, srv1937549354=49, srv188169842=44, srv1273773142=14, srv1321974888=18, srv2131966266=56, srv1182984431=8, srv932219680=99, srv1803455787=42, srv392854594=72, srv1888199607=45, srv1950114482=51, srv709759951=89, srv2147398586=58, srv598409157=80, srv1505916535=27, srv242708308=60, srv1715664882=36, srv1308153942=17, srv1940190747=50, srv1279617782=15, srv1863701352=43, srv243430031=61, srv1712966335=35, srv594841708=79, srv1250813762=12, srv857142480=95, srv256012864=63, srv256642=64, srv193339352=47, srv1608605212=30, srv1672893819=32, srv1661768674=31, srv1178371048=7, srv1262374980=13, srv1751404572=38, srv1934195587=48, srv1119148680=4, srv1191985913=11, srv885435734=96, srv1293852394=16, srv244054924=62, srv634733402=83, srv1102250671=3, srv277115918=66, srv699715809=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:24,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:24,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:24,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:24,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:24,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:24,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-10 00:53:24,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-10 00:53:24,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-10 00:53:24,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-10 00:53:24,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-10 00:53:24,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:24,123 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1608605212:83 , srv1661768674:83 , srv1693533622:83 , srv1779040107:83 , srv191412149:83 , srv333406664:83 , srv363710411:83 , srv522929612:83 , srv736463011:83 , srv857142480:83 , srv1037156773:84 , srv10444358:84 , srv1075414181:84 , srv1102250671:84 , srv1119148680:84 , srv1164108141:84 , srv1174493252:84 , srv1178371048:84 , srv1182984431:84 , srv1188989733:84 , srv1190940957:84 , srv1191985913:84 , srv1250813762:84 , srv1273773142:84 , srv1279617782:84 , srv1293852394:84 , srv1308153942:84 , srv1321974888:84 , srv1343054369:84 , srv1387106497:84 , srv1395633367:84 , srv1400878673:84 , srv1407056356:84 , srv1408499621:84 , srv14593139:84 , srv1505916535:84 , srv1506698678:84 , srv1672893819:84 , srv1689569626:84 , srv1712966335:84 , srv1715664882:84 , srv1751404572:84 , srv1767562723:84 , srv1792802097:84 , srv1803455787:84 , srv1863701352:84 , srv188169842:84 , srv1888199607:84 , srv1934195587:84 , srv1937549354:84 , srv1940190747:84 , srv1950114482:84 , srv2007181186:84 , srv2060078515:84 , srv2066283342:84 , srv2131966266:84 , srv2141192270:84 , srv2147398586:84 , srv230861697:84 , srv242708308:84 , srv244054924:84 , srv256012864:84 , srv256642:84 , srv263414787:84 , srv32756278:84 , srv340293031:84 , srv387823670:84 , srv392854594:84 , srv481441539:84 , srv485453467:84 , srv528026438:84 , srv529231342:84 , srv594841708:84 , srv598409157:84 , srv612870911:84 , srv634733402:84 , srv637009635:84 , srv638052247:84 , srv653433066:84 , srv665735732:84 , srv699715809:84 , srv709759951:84 , srv745027173:84 , srv811948993:84 , srv815823814:84 , srv822832817:84 , srv901256406:84 , srv932219680:84 , srv1262374980:85 , srv1440571341:85 , srv1587823610:85 , srv193339352:85 , srv211471933:85 , srv243430031:85 , srv277115918:85 , srv591505995:85 , srv603767746:85 , srv885435734:85 , srv894588696:85 , srv1731913698:683 }
2024-04-10 00:53:24,148 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=360 (was 360), ProcessCount=169 (was 169), AvailableMemoryMB=15723 (was 16477)
2024-04-10 00:53:24,157 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=360, ProcessCount=169, AvailableMemoryMB=15723
2024-04-10 00:53:24,158 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-10 00:53:24,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-10 00:53:24,173 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv104426932:84 , srv1077416890:84 , srv1319272825:84 , srv1330071964:84 , srv1369821326:84 , srv1370833176:84 , srv1459138901:84 , srv1491621712:84 , srv1522237221:84 , srv1631749653:84 , srv1652830540:84 , srv1756953617:84 , srv1768882832:84 , srv178114695:84 , srv178341599:84 , srv1845951631:84 , srv2086883918:84 , srv253913:84 , srv265804606:84 , srv296538529:84 , srv514324331:84 , srv66505688:84 , srv687437885:84 , srv769482615:84 , srv803307747:84 , srv812903528:84 , srv852591784:84 , srv864991375:84 , srv93328459:84 , srv291617639:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-10 00:53:24,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:53:24,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1369821326=4, srv1330071964=3, srv864991375=28, srv178114695=13, srv265804606=18, srv812903528=26, srv66505688=22, srv1631749653=9, srv93328459=29, srv769482615=24, srv104426932=0, srv514324331=21, srv1459138901=6, srv2086883918=16, srv1652830540=10, srv1077416890=1, srv1845951631=15, srv1319272825=2, srv178341599=14, srv291617639=19, srv687437885=23, srv296538529=20, srv803307747=25, srv1768882832=12, srv1756953617=11, srv253913=17, srv852591784=27, srv1491621712=7, srv1370833176=5, srv1522237221=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:53:24,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:53:24,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-10 00:53:24,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.11234149171835, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:53:24,203 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912547027636398, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.11234149171835, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-10 00:55:24,190 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5541482 different iterations.  Found a solution that moves 1863 regions; Going from a computed imbalance of 0.9912547027636398 to a new imbalance of 4.3483349388262195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.69, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:55:24,197 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv104426932:90 , srv1077416890:90 , srv1319272825:90 , srv1330071964:90 , srv1369821326:90 , srv1370833176:90 , srv1459138901:90 , srv1491621712:90 , srv1522237221:90 , srv1631749653:90 , srv1652830540:90 , srv1756953617:90 , srv1768882832:90 , srv178114695:90 , srv178341599:90 , srv1845951631:90 , srv2086883918:90 , srv253913:90 , srv265804606:90 , srv291617639:90 , srv296538529:90 , srv514324331:90 , srv66505688:90 , srv687437885:90 , srv769482615:90 , srv803307747:90 , srv812903528:90 , srv852591784:90 , srv864991375:90 , srv93328459:90 }
2024-04-10 00:55:24,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-10 00:55:24,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1369821326=4, srv1330071964=3, srv864991375=28, srv178114695=13, srv265804606=18, srv812903528=26, srv66505688=22, srv1631749653=9, srv93328459=29, srv769482615=24, srv104426932=0, srv514324331=21, srv1459138901=6, srv2086883918=16, srv1652830540=10, srv1077416890=1, srv1845951631=15, srv1319272825=2, srv178341599=14, srv291617639=19, srv687437885=23, srv296538529=20, srv803307747=25, srv1768882832=12, srv1756953617=11, srv253913=17, srv852591784=27, srv1491621712=7, srv1370833176=5, srv1522237221=8} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-10 00:55:24,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-10 00:55:24,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-10 00:55:24,223 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-10 00:55:24,223 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv104426932:90 , srv1077416890:90 , srv1319272825:90 , srv1330071964:90 , srv1369821326:90 , srv1370833176:90 , srv1459138901:90 , srv1491621712:90 , srv1522237221:90 , srv1631749653:90 , srv1652830540:90 , srv1756953617:90 , srv1768882832:90 , srv178114695:90 , srv178341599:90 , srv1845951631:90 , srv2086883918:90 , srv253913:90 , srv265804606:90 , srv291617639:90 , srv296538529:90 , srv514324331:90 , srv66505688:90 , srv687437885:90 , srv769482615:90 , srv803307747:90 , srv812903528:90 , srv852591784:90 , srv864991375:90 , srv93328459:90 }
2024-04-10 00:55:24,241 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=263 (was 360), ProcessCount=169 (was 169), AvailableMemoryMB=14967 (was 15723)