2023-12-03 01:53:15,650 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-03 01:53:15,972 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-03 01:53:16,091 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-03 01:53:16,160 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-03 01:53:16,169 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-03 01:53:16,184 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=171, AvailableMemoryMB=4494
2023-12-03 01:53:16,186 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-03 01:53:16,186 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-03 01:53:16,323 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100593869:84 , srv1013191335:84 , srv1052180:84 , srv108159467:84 , srv1090591190:84 , srv1090750987:84 , srv1117000664:84 , srv112290624:84 , srv1133691823:84 , srv1137207408:84 , srv1145951442:84 , srv1159555493:84 , srv1200346497:84 , srv1236323971:84 , srv1272688540:84 , srv1302821940:84 , srv1307017549:84 , srv1326597687:84 , srv134406112:84 , srv1360401371:84 , srv1373641917:84 , srv1379549153:84 , srv1383447446:84 , srv1383876700:84 , srv1406840849:84 , srv1409728363:84 , srv1416020443:84 , srv1435561090:84 , srv1447735219:84 , srv1450245285:84 , srv1459632148:84 , srv1521529499:84 , srv1546129262:84 , srv1555064030:84 , srv1563928992:84 , srv1626438469:84 , srv1632216704:84 , srv1638155965:84 , srv164366309:84 , srv1647237325:84 , srv1658990786:84 , srv1661168404:84 , srv166702269:84 , srv1686209404:84 , srv1724497489:84 , srv172660055:84 , srv1763654286:84 , srv1780504458:84 , srv1819286024:84 , srv1819679584:84 , srv1822482829:84 , srv184199571:84 , srv1850274335:84 , srv1852620908:84 , srv1874484378:84 , srv190580017:84 , srv19081645:84 , srv1948579075:84 , srv1960150666:84 , srv1963560130:84 , srv2057196735:84 , srv2125979931:84 , srv2140385339:84 , srv222918581:84 , srv23160071:84 , srv244954199:84 , srv285103767:84 , srv289890849:84 , srv315615650:84 , srv316009065:84 , srv318865744:84 , srv341741578:84 , srv362621634:84 , srv36957172:84 , srv374384903:84 , srv399517826:84 , srv413638941:84 , srv449534167:84 , srv456530514:84 , srv493159173:84 , srv50347900:84 , srv548186858:84 , srv559888914:84 , srv566517562:84 , srv593662610:84 , srv595058338:84 , srv629916596:84 , srv644795438:84 , srv651484762:84 , srv670894490:84 , srv71355968:84 , srv716553446:84 , srv716696083:84 , srv791541526:84 , srv816899058:84 , srv92361766:84 , srv932274301:84 , srv963211237:84 , srv984519195:84 , srv108727616:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-03 01:53:16,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1236323971=14, srv100593869=0, srv1133691823=9, srv1521529499=32, srv1555064030=34, srv166702269=43, srv1819286024=49, srv1963560130=60, srv984519195=99, srv1326597687=18, srv670894490=90, srv963211237=98, srv374384903=75, srv1724497489=45, srv1145951442=11, srv108727616=4, srv1373641917=21, srv1780504458=48, srv1013191335=1, srv172660055=46, srv1647237325=40, srv1638155965=38, srv1852620908=54, srv164366309=39, srv1090591190=5, srv1450245285=30, srv651484762=89, srv593662610=85, srv1850274335=53, srv595058338=86, srv285103767=67, srv112290624=8, srv1763654286=47, srv289890849=68, srv644795438=88, srv1159555493=12, srv184199571=52, srv1626438469=36, srv362621634=73, srv413638941=77, srv50347900=81, srv1383447446=23, srv1661168404=42, srv71355968=91, srv566517562=84, srv629916596=87, srv1822482829=51, srv1632216704=37, srv456530514=79, srv1137207408=10, srv791541526=94, srv23160071=65, srv399517826=76, srv1459632148=31, srv1658990786=41, srv1409728363=26, srv1416020443=27, srv341741578=72, srv1435561090=28, srv1379549153=22, srv1272688540=15, srv716553446=92, srv318865744=71, srv315615650=69, srv1052180=2, srv2140385339=63, srv244954199=66, srv108159467=3, srv1447735219=29, srv1686209404=44, srv92361766=96, srv190580017=56, srv1200346497=13, srv1090750987=6, srv548186858=82, srv1360401371=20, srv716696083=93, srv449534167=78, srv493159173=80, srv932274301=97, srv222918581=64, srv1406840849=25, srv2125979931=62, srv1302821940=16, srv1819679584=50, srv1563928992=35, srv36957172=74, srv2057196735=61, srv19081645=57, srv134406112=19, srv1948579075=58, srv316009065=70, srv1874484378=55, srv559888914=83, srv816899058=95, srv1307017549=17, srv1546129262=33, srv1960150666=59, srv1383876700=24, srv1117000664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:16,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:16,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:16,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:16,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:16,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:16,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:16,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:16,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:16,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-03 01:53:16,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-03 01:53:16,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-03 01:53:16,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-03 01:53:16,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-03 01:53:16,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-03 01:53:16,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-03 01:53:16,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-03 01:53:16,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-03 01:53:16,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-03 01:53:16,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-03 01:53:16,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-03 01:53:16,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-03 01:53:16,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-03 01:53:16,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-03 01:53:16,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-03 01:53:16,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-03 01:53:16,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-03 01:53:16,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-03 01:53:16,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-03 01:53:16,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:16,732 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-03 01:53:16,732 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-03 01:53:17,627 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1171 ms to try 5000 different iterations.  Found a solution that moves 4683 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17963716718066874. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03870036072105972, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06008785188274979, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5203333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03870036072105972, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:17,642 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1117000664:81 , srv1145951442:82 , srv1416020443:82 , srv1406840849:83 , srv1450245285:83 , srv2057196735:83 , srv670894490:83 , srv816899058:83 , srv1090591190:84 , srv1326597687:84 , srv1360401371:84 , srv1373641917:84 , srv1383447446:84 , srv1409728363:84 , srv19081645:84 , srv1948579075:84 , srv318865744:84 , srv456530514:84 , srv50347900:84 , srv593662610:84 , srv629916596:84 , srv651484762:84 , srv963211237:84 , srv984519195:84 , srv1052180:85 , srv1546129262:85 , srv1632216704:85 , srv1686209404:85 , srv1822482829:85 , srv1852620908:85 , srv244954199:85 , srv595058338:85 , srv716696083:85 , srv1013191335:86 , srv112290624:86 , srv1133691823:86 , srv1137207408:86 , srv1200346497:86 , srv1272688540:86 , srv1447735219:86 , srv164366309:86 , srv1658990786:86 , srv1661168404:86 , srv1780504458:86 , srv1850274335:86 , srv1874484378:86 , srv190580017:86 , srv2125979931:86 , srv315615650:86 , srv449534167:86 , srv493159173:86 , srv791541526:86 , srv932274301:86 , srv108159467:87 , srv1159555493:87 , srv1379549153:87 , srv1521529499:87 , srv1555064030:87 , srv1647237325:87 , srv1819286024:87 , srv23160071:87 , srv316009065:87 , srv362621634:87 , srv36957172:87 , srv559888914:87 , srv566517562:87 , srv716553446:87 , srv1236323971:88 , srv1459632148:88 , srv1563928992:88 , srv1626438469:88 , srv1763654286:88 , srv1819679584:88 , srv1960150666:88 , srv341741578:88 , srv413638941:88 , srv100593869:89 , srv1090750987:89 , srv1638155965:89 , srv1724497489:89 , srv184199571:89 , srv2140385339:89 , srv222918581:89 , srv399517826:89 , srv548186858:89 , srv644795438:89 , srv92361766:89 , srv1307017549:90 , srv134406112:90 , srv1383876700:90 , srv1435561090:90 , srv166702269:90 , srv172660055:90 , srv285103767:90 , srv374384903:90 , srv289890849:91 , srv1963560130:92 , srv71355968:92 , srv1302821940:93 , srv108727616:434 }
2023-12-03 01:53:17,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:17,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1236323971=14, srv100593869=0, srv1133691823=9, srv1521529499=32, srv1555064030=34, srv166702269=43, srv1819286024=49, srv1963560130=60, srv984519195=99, srv1326597687=18, srv670894490=90, srv963211237=98, srv374384903=75, srv1724497489=45, srv1145951442=11, srv108727616=4, srv1373641917=21, srv1780504458=48, srv1013191335=1, srv172660055=46, srv1647237325=40, srv1638155965=38, srv1852620908=54, srv164366309=39, srv1090591190=5, srv1450245285=30, srv651484762=89, srv593662610=85, srv1850274335=53, srv595058338=86, srv285103767=67, srv112290624=8, srv1763654286=47, srv289890849=68, srv644795438=88, srv1159555493=12, srv184199571=52, srv1626438469=36, srv362621634=73, srv413638941=77, srv50347900=81, srv1383447446=23, srv1661168404=42, srv71355968=91, srv566517562=84, srv629916596=87, srv1822482829=51, srv1632216704=37, srv456530514=79, srv1137207408=10, srv791541526=94, srv23160071=65, srv399517826=76, srv1459632148=31, srv1658990786=41, srv1409728363=26, srv1416020443=27, srv341741578=72, srv1435561090=28, srv1379549153=22, srv1272688540=15, srv716553446=92, srv318865744=71, srv315615650=69, srv1052180=2, srv2140385339=63, srv244954199=66, srv108159467=3, srv1447735219=29, srv1686209404=44, srv92361766=96, srv190580017=56, srv1200346497=13, srv1090750987=6, srv548186858=82, srv1360401371=20, srv716696083=93, srv449534167=78, srv493159173=80, srv932274301=97, srv222918581=64, srv1406840849=25, srv2125979931=62, srv1302821940=16, srv1819679584=50, srv1563928992=35, srv36957172=74, srv2057196735=61, srv19081645=57, srv134406112=19, srv1948579075=58, srv316009065=70, srv1874484378=55, srv559888914=83, srv816899058=95, srv1307017549=17, srv1546129262=33, srv1960150666=59, srv1383876700=24, srv1117000664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:17,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:17,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:17,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:17,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:17,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-03 01:53:17,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-03 01:53:17,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-03 01:53:17,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-03 01:53:17,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-03 01:53:17,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-03 01:53:17,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-03 01:53:17,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-03 01:53:17,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-03 01:53:17,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-03 01:53:17,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03870036072105972, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06008785188274979, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03870036072105972, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:17,852 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-03 01:53:17,853 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17960437611381122, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03870036072105972, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06008785188274979, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03870036072105972, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-03 01:53:18,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 489 ms to try 5000 different iterations.  Found a solution that moves 2645 regions; Going from a computed imbalance of 0.17960437611381122 to a new imbalance of 0.01659708428766092. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020478744466752116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036081961817155055, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29388888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020478744466752116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04875, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:18,212 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1963560130:78 , srv1302821940:79 , srv71355968:79 , srv172660055:80 , srv1307017549:81 , srv166702269:81 , srv1090750987:82 , srv1638155965:82 , srv374384903:82 , srv413638941:82 , srv1159555493:83 , srv1383876700:83 , srv1724497489:83 , srv1763654286:83 , srv184199571:83 , srv285103767:83 , srv548186858:83 , srv644795438:83 , srv100593869:84 , srv108159467:84 , srv1236323971:84 , srv1272688540:84 , srv134406112:84 , srv1379549153:84 , srv1435561090:84 , srv1459632148:84 , srv1555064030:84 , srv1563928992:84 , srv1626438469:84 , srv1819679584:84 , srv1960150666:84 , srv2140385339:84 , srv222918581:84 , srv289890849:84 , srv341741578:84 , srv399517826:84 , srv92361766:84 , srv112290624:85 , srv1632216704:85 , srv1658990786:85 , srv190580017:85 , srv316009065:85 , srv493159173:85 , srv932274301:85 , srv1133691823:86 , srv1373641917:86 , srv1521529499:86 , srv164366309:86 , srv1686209404:86 , srv1822482829:86 , srv1852620908:86 , srv1874484378:86 , srv2125979931:86 , srv23160071:86 , srv362621634:86 , srv36957172:86 , srv559888914:86 , srv566517562:86 , srv716696083:86 , srv984519195:86 , srv1013191335:87 , srv1052180:87 , srv1137207408:87 , srv1200346497:87 , srv1406840849:87 , srv1546129262:87 , srv1647237325:87 , srv1661168404:87 , srv1780504458:87 , srv1850274335:87 , srv1948579075:87 , srv50347900:87 , srv651484762:87 , srv716553446:87 , srv791541526:87 , srv1326597687:88 , srv1383447446:88 , srv1450245285:88 , srv1819286024:88 , srv19081645:88 , srv244954199:88 , srv449534167:88 , srv593662610:88 , srv963211237:88 , srv1090591190:89 , srv1145951442:89 , srv1409728363:89 , srv1416020443:89 , srv1447735219:89 , srv2057196735:89 , srv315615650:89 , srv595058338:89 , srv816899058:89 , srv1117000664:90 , srv1360401371:90 , srv318865744:90 , srv456530514:90 , srv629916596:91 , srv670894490:92 , srv108727616:522 }
2023-12-03 01:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:18,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1236323971=14, srv100593869=0, srv1133691823=9, srv1521529499=32, srv1555064030=34, srv166702269=43, srv1819286024=49, srv1963560130=60, srv984519195=99, srv1326597687=18, srv670894490=90, srv963211237=98, srv374384903=75, srv1724497489=45, srv1145951442=11, srv108727616=4, srv1373641917=21, srv1780504458=48, srv1013191335=1, srv172660055=46, srv1647237325=40, srv1638155965=38, srv1852620908=54, srv164366309=39, srv1090591190=5, srv1450245285=30, srv651484762=89, srv593662610=85, srv1850274335=53, srv595058338=86, srv285103767=67, srv112290624=8, srv1763654286=47, srv289890849=68, srv644795438=88, srv1159555493=12, srv184199571=52, srv1626438469=36, srv362621634=73, srv413638941=77, srv50347900=81, srv1383447446=23, srv1661168404=42, srv71355968=91, srv566517562=84, srv629916596=87, srv1822482829=51, srv1632216704=37, srv456530514=79, srv1137207408=10, srv791541526=94, srv23160071=65, srv399517826=76, srv1459632148=31, srv1658990786=41, srv1409728363=26, srv1416020443=27, srv341741578=72, srv1435561090=28, srv1379549153=22, srv1272688540=15, srv716553446=92, srv318865744=71, srv315615650=69, srv1052180=2, srv2140385339=63, srv244954199=66, srv108159467=3, srv1447735219=29, srv1686209404=44, srv92361766=96, srv190580017=56, srv1200346497=13, srv1090750987=6, srv548186858=82, srv1360401371=20, srv716696083=93, srv449534167=78, srv493159173=80, srv932274301=97, srv222918581=64, srv1406840849=25, srv2125979931=62, srv1302821940=16, srv1819679584=50, srv1563928992=35, srv36957172=74, srv2057196735=61, srv19081645=57, srv134406112=19, srv1948579075=58, srv316009065=70, srv1874484378=55, srv559888914=83, srv816899058=95, srv1307017549=17, srv1546129262=33, srv1960150666=59, srv1383876700=24, srv1117000664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-03 01:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-03 01:53:18,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-03 01:53:18,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-03 01:53:18,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-03 01:53:18,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-03 01:53:18,345 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020478744466752116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036081961817155055, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020478744466752116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04875, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:18,345 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-03 01:53:18,345 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016578563601810366, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020478744466752116); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.036081961817155055, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020478744466752116); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01325); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04875, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-03 01:53:18,748 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 499 ms to try 5000 different iterations.  Found a solution that moves 985 regions; Going from a computed imbalance of 0.016578563601810366 to a new imbalance of 6.520587686268625E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.10944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:18,750 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv629916596:82 , srv1435561090:83 , srv1447735219:83 , srv1819286024:83 , srv222918581:83 , srv315615650:83 , srv456530514:83 , srv716553446:83 , srv1013191335:84 , srv1090591190:84 , srv1133691823:84 , srv1137207408:84 , srv1200346497:84 , srv134406112:84 , srv1383447446:84 , srv1409728363:84 , srv1521529499:84 , srv1647237325:84 , srv1661168404:84 , srv1780504458:84 , srv289890849:84 , srv318865744:84 , srv362621634:84 , srv449534167:84 , srv559888914:84 , srv595058338:84 , srv670894490:84 , srv816899058:84 , srv92361766:84 , srv100593869:85 , srv1052180:85 , srv1302821940:85 , srv1360401371:85 , srv1383876700:85 , srv1459632148:85 , srv1626438469:85 , srv164366309:85 , srv1724497489:85 , srv184199571:85 , srv1850274335:85 , srv1874484378:85 , srv19081645:85 , srv1960150666:85 , srv2140385339:85 , srv23160071:85 , srv244954199:85 , srv285103767:85 , srv316009065:85 , srv341741578:85 , srv36957172:85 , srv399517826:85 , srv566517562:85 , srv593662610:85 , srv644795438:85 , srv108159467:86 , srv112290624:86 , srv1145951442:86 , srv1236323971:86 , srv1326597687:86 , srv1406840849:86 , srv1416020443:86 , srv1546129262:86 , srv1555064030:86 , srv1563928992:86 , srv1638155965:86 , srv1658990786:86 , srv166702269:86 , srv1686209404:86 , srv1763654286:86 , srv1819679584:86 , srv1852620908:86 , srv190580017:86 , srv2057196735:86 , srv2125979931:86 , srv374384903:86 , srv493159173:86 , srv50347900:86 , srv548186858:86 , srv651484762:86 , srv71355968:86 , srv716696083:86 , srv791541526:86 , srv932274301:86 , srv1090750987:87 , srv1117000664:87 , srv1159555493:87 , srv1307017549:87 , srv1373641917:87 , srv1379549153:87 , srv1632216704:87 , srv172660055:87 , srv1822482829:87 , srv1948579075:87 , srv413638941:87 , srv984519195:87 , srv1272688540:88 , srv1450245285:88 , srv1963560130:88 , srv963211237:88 , srv108727616:558 }
2023-12-03 01:53:18,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:18,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1236323971=14, srv100593869=0, srv1133691823=9, srv1521529499=32, srv1555064030=34, srv166702269=43, srv1819286024=49, srv1963560130=60, srv984519195=99, srv1326597687=18, srv670894490=90, srv963211237=98, srv374384903=75, srv1724497489=45, srv1145951442=11, srv108727616=4, srv1373641917=21, srv1780504458=48, srv1013191335=1, srv172660055=46, srv1647237325=40, srv1638155965=38, srv1852620908=54, srv164366309=39, srv1090591190=5, srv1450245285=30, srv651484762=89, srv593662610=85, srv1850274335=53, srv595058338=86, srv285103767=67, srv112290624=8, srv1763654286=47, srv289890849=68, srv644795438=88, srv1159555493=12, srv184199571=52, srv1626438469=36, srv362621634=73, srv413638941=77, srv50347900=81, srv1383447446=23, srv1661168404=42, srv71355968=91, srv566517562=84, srv629916596=87, srv1822482829=51, srv1632216704=37, srv456530514=79, srv1137207408=10, srv791541526=94, srv23160071=65, srv399517826=76, srv1459632148=31, srv1658990786=41, srv1409728363=26, srv1416020443=27, srv341741578=72, srv1435561090=28, srv1379549153=22, srv1272688540=15, srv716553446=92, srv318865744=71, srv315615650=69, srv1052180=2, srv2140385339=63, srv244954199=66, srv108159467=3, srv1447735219=29, srv1686209404=44, srv92361766=96, srv190580017=56, srv1200346497=13, srv1090750987=6, srv548186858=82, srv1360401371=20, srv716696083=93, srv449534167=78, srv493159173=80, srv932274301=97, srv222918581=64, srv1406840849=25, srv2125979931=62, srv1302821940=16, srv1819679584=50, srv1563928992=35, srv36957172=74, srv2057196735=61, srv19081645=57, srv134406112=19, srv1948579075=58, srv316009065=70, srv1874484378=55, srv559888914=83, srv816899058=95, srv1307017549=17, srv1546129262=33, srv1960150666=59, srv1383876700=24, srv1117000664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-03 01:53:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-03 01:53:18,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-03 01:53:18,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-03 01:53:18,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-03 01:53:18,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-03 01:53:18,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-03 01:53:18,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-03 01:53:18,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-03 01:53:18,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-03 01:53:18,900 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:18,900 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-03 01:53:18,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.451616512118161E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006334800074315197); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006334800074315197); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004583333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-03 01:53:19,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 392 ms to try 5000 different iterations.  Found a solution that moves 298 regions; Going from a computed imbalance of 6.451616512118161E-4 to a new imbalance of 2.1864224041095595E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=6.70025210172808E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.03311111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=6.70025210172808E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:19,179 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv289890849:82 , srv100593869:83 , srv1360401371:83 , srv1450245285:83 , srv399517826:83 , srv566517562:83 , srv670894490:83 , srv791541526:83 , srv963211237:83 , srv1090750987:84 , srv1117000664:84 , srv1145951442:84 , srv1159555493:84 , srv1236323971:84 , srv1272688540:84 , srv1307017549:84 , srv1326597687:84 , srv134406112:84 , srv1383876700:84 , srv1447735219:84 , srv1546129262:84 , srv1647237325:84 , srv1724497489:84 , srv172660055:84 , srv1780504458:84 , srv1819286024:84 , srv1819679584:84 , srv1822482829:84 , srv1850274335:84 , srv190580017:84 , srv1948579075:84 , srv1963560130:84 , srv2057196735:84 , srv2140385339:84 , srv222918581:84 , srv285103767:84 , srv315615650:84 , srv318865744:84 , srv341741578:84 , srv36957172:84 , srv374384903:84 , srv449534167:84 , srv493159173:84 , srv595058338:84 , srv644795438:84 , srv984519195:84 , srv1013191335:85 , srv1090591190:85 , srv112290624:85 , srv1137207408:85 , srv1200346497:85 , srv1302821940:85 , srv1373641917:85 , srv1379549153:85 , srv1409728363:85 , srv1416020443:85 , srv1435561090:85 , srv1459632148:85 , srv1521529499:85 , srv1555064030:85 , srv1563928992:85 , srv1626438469:85 , srv1632216704:85 , srv1658990786:85 , srv166702269:85 , srv1686209404:85 , srv1763654286:85 , srv184199571:85 , srv1852620908:85 , srv1874484378:85 , srv2125979931:85 , srv23160071:85 , srv244954199:85 , srv316009065:85 , srv362621634:85 , srv413638941:85 , srv456530514:85 , srv50347900:85 , srv548186858:85 , srv559888914:85 , srv593662610:85 , srv651484762:85 , srv71355968:85 , srv716696083:85 , srv92361766:85 , srv932274301:85 , srv1052180:86 , srv108159467:86 , srv1133691823:86 , srv1383447446:86 , srv1406840849:86 , srv1638155965:86 , srv164366309:86 , srv1661168404:86 , srv19081645:86 , srv1960150666:86 , srv629916596:86 , srv716553446:86 , srv816899058:87 , srv108727616:627 }
2023-12-03 01:53:19,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:19,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1236323971=14, srv100593869=0, srv1133691823=9, srv1521529499=32, srv1555064030=34, srv166702269=43, srv1819286024=49, srv1963560130=60, srv984519195=99, srv1326597687=18, srv670894490=90, srv963211237=98, srv374384903=75, srv1724497489=45, srv1145951442=11, srv108727616=4, srv1373641917=21, srv1780504458=48, srv1013191335=1, srv172660055=46, srv1647237325=40, srv1638155965=38, srv1852620908=54, srv164366309=39, srv1090591190=5, srv1450245285=30, srv651484762=89, srv593662610=85, srv1850274335=53, srv595058338=86, srv285103767=67, srv112290624=8, srv1763654286=47, srv289890849=68, srv644795438=88, srv1159555493=12, srv184199571=52, srv1626438469=36, srv362621634=73, srv413638941=77, srv50347900=81, srv1383447446=23, srv1661168404=42, srv71355968=91, srv566517562=84, srv629916596=87, srv1822482829=51, srv1632216704=37, srv456530514=79, srv1137207408=10, srv791541526=94, srv23160071=65, srv399517826=76, srv1459632148=31, srv1658990786=41, srv1409728363=26, srv1416020443=27, srv341741578=72, srv1435561090=28, srv1379549153=22, srv1272688540=15, srv716553446=92, srv318865744=71, srv315615650=69, srv1052180=2, srv2140385339=63, srv244954199=66, srv108159467=3, srv1447735219=29, srv1686209404=44, srv92361766=96, srv190580017=56, srv1200346497=13, srv1090750987=6, srv548186858=82, srv1360401371=20, srv716696083=93, srv449534167=78, srv493159173=80, srv932274301=97, srv222918581=64, srv1406840849=25, srv2125979931=62, srv1302821940=16, srv1819679584=50, srv1563928992=35, srv36957172=74, srv2057196735=61, srv19081645=57, srv134406112=19, srv1948579075=58, srv316009065=70, srv1874484378=55, srv559888914=83, srv816899058=95, srv1307017549=17, srv1546129262=33, srv1960150666=59, srv1383876700=24, srv1117000664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:19,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-03 01:53:19,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-03 01:53:19,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-03 01:53:19,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-03 01:53:19,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-03 01:53:19,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-03 01:53:19,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-03 01:53:19,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-03 01:53:19,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-03 01:53:19,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-03 01:53:19,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-03 01:53:19,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-03 01:53:19,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-03 01:53:19,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.9777583442431806E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=6.70025210172808E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=6.70025210172808E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:19,363 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv289890849:82 , srv100593869:83 , srv1360401371:83 , srv1450245285:83 , srv399517826:83 , srv566517562:83 , srv670894490:83 , srv791541526:83 , srv963211237:83 , srv1090750987:84 , srv1117000664:84 , srv1145951442:84 , srv1159555493:84 , srv1236323971:84 , srv1272688540:84 , srv1307017549:84 , srv1326597687:84 , srv134406112:84 , srv1383876700:84 , srv1447735219:84 , srv1546129262:84 , srv1647237325:84 , srv1724497489:84 , srv172660055:84 , srv1780504458:84 , srv1819286024:84 , srv1819679584:84 , srv1822482829:84 , srv1850274335:84 , srv190580017:84 , srv1948579075:84 , srv1963560130:84 , srv2057196735:84 , srv2140385339:84 , srv222918581:84 , srv285103767:84 , srv315615650:84 , srv318865744:84 , srv341741578:84 , srv36957172:84 , srv374384903:84 , srv449534167:84 , srv493159173:84 , srv595058338:84 , srv644795438:84 , srv984519195:84 , srv1013191335:85 , srv1090591190:85 , srv112290624:85 , srv1137207408:85 , srv1200346497:85 , srv1302821940:85 , srv1373641917:85 , srv1379549153:85 , srv1409728363:85 , srv1416020443:85 , srv1435561090:85 , srv1459632148:85 , srv1521529499:85 , srv1555064030:85 , srv1563928992:85 , srv1626438469:85 , srv1632216704:85 , srv1658990786:85 , srv166702269:85 , srv1686209404:85 , srv1763654286:85 , srv184199571:85 , srv1852620908:85 , srv1874484378:85 , srv2125979931:85 , srv23160071:85 , srv244954199:85 , srv316009065:85 , srv362621634:85 , srv413638941:85 , srv456530514:85 , srv50347900:85 , srv548186858:85 , srv559888914:85 , srv593662610:85 , srv651484762:85 , srv71355968:85 , srv716696083:85 , srv92361766:85 , srv932274301:85 , srv1052180:86 , srv108159467:86 , srv1133691823:86 , srv1383447446:86 , srv1406840849:86 , srv1638155965:86 , srv164366309:86 , srv1661168404:86 , srv19081645:86 , srv1960150666:86 , srv629916596:86 , srv716553446:86 , srv816899058:87 , srv108727616:627 }
2023-12-03 01:53:19,402 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=279 (was 250) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=3571 (was 4494)
2023-12-03 01:53:19,413 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=171, AvailableMemoryMB=3570
2023-12-03 01:53:19,414 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-03 01:53:19,414 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-03 01:53:19,428 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1320131938:84 , srv1356593967:84 , srv1470636543:84 , srv1543630491:84 , srv1574482486:84 , srv1588336230:84 , srv1716658369:84 , srv1789144534:84 , srv1891969004:84 , srv1919763687:84 , srv1988453809:84 , srv2044411656:84 , srv206103900:84 , srv2132234242:84 , srv235762464:84 , srv27041516:84 , srv271609856:84 , srv36138971:84 , srv504370722:84 , srv517247436:84 , srv631153430:84 , srv662146753:84 , srv673568705:84 , srv781089827:84 , srv794726238:84 , srv801518676:84 , srv829755469:84 , srv902684655:84 , srv991117190:84 , srv1333612351:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-03 01:53:19,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:53:19,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv27041516=16, srv1988453809=11, srv517247436=20, srv1716658369=7, srv504370722=19, srv673568705=23, srv235762464=15, srv991117190=29, srv1574482486=5, srv206103900=13, srv1891969004=9, srv781089827=24, srv271609856=17, srv1470636543=3, srv1789144534=8, srv631153430=21, srv662146753=22, srv801518676=26, srv1356593967=2, srv1333612351=1, srv2044411656=12, srv1543630491=4, srv2132234242=14, srv902684655=28, srv36138971=18, srv794726238=25, srv829755469=27, srv1320131938=0, srv1588336230=6, srv1919763687=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:53:19,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:53:19,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:53:19,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:53:19,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:53:19,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-03 01:53:19,473 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1724454634367913, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:53:19,473 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912736413288705, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1724454634367913, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-03 01:55:19,453 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6296589 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912736413288705 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:55:19,458 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1320131938:90 , srv1333612351:90 , srv1356593967:90 , srv1470636543:90 , srv1543630491:90 , srv1574482486:90 , srv1588336230:90 , srv1716658369:90 , srv1789144534:90 , srv1891969004:90 , srv1919763687:90 , srv1988453809:90 , srv2044411656:90 , srv206103900:90 , srv2132234242:90 , srv235762464:90 , srv27041516:90 , srv271609856:90 , srv36138971:90 , srv504370722:90 , srv517247436:90 , srv631153430:90 , srv662146753:90 , srv673568705:90 , srv781089827:90 , srv794726238:90 , srv801518676:90 , srv829755469:90 , srv902684655:90 , srv991117190:90 }
2023-12-03 01:55:19,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-03 01:55:19,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv27041516=16, srv1988453809=11, srv517247436=20, srv1716658369=7, srv504370722=19, srv673568705=23, srv235762464=15, srv991117190=29, srv1574482486=5, srv206103900=13, srv1891969004=9, srv781089827=24, srv271609856=17, srv1470636543=3, srv1789144534=8, srv631153430=21, srv662146753=22, srv801518676=26, srv1356593967=2, srv1333612351=1, srv2044411656=12, srv1543630491=4, srv2132234242=14, srv902684655=28, srv36138971=18, srv794726238=25, srv829755469=27, srv1320131938=0, srv1588336230=6, srv1919763687=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-03 01:55:19,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-03 01:55:19,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-03 01:55:19,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-03 01:55:19,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-03 01:55:19,476 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-03 01:55:19,477 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1320131938:90 , srv1333612351:90 , srv1356593967:90 , srv1470636543:90 , srv1543630491:90 , srv1574482486:90 , srv1588336230:90 , srv1716658369:90 , srv1789144534:90 , srv1891969004:90 , srv1919763687:90 , srv1988453809:90 , srv2044411656:90 , srv206103900:90 , srv2132234242:90 , srv235762464:90 , srv27041516:90 , srv271609856:90 , srv36138971:90 , srv504370722:90 , srv517247436:90 , srv631153430:90 , srv662146753:90 , srv673568705:90 , srv781089827:90 , srv794726238:90 , srv801518676:90 , srv829755469:90 , srv902684655:90 , srv991117190:90 }
2023-12-03 01:55:19,489 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=141 (was 279), ProcessCount=168 (was 171), AvailableMemoryMB=2994 (was 3570)