2023-10-11 05:53:28,674 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-11 05:53:28,966 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-11 05:53:29,082 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-11 05:53:29,148 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-11 05:53:29,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-11 05:53:29,169 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=276, ProcessCount=171, AvailableMemoryMB=7799
2023-10-11 05:53:29,171 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-11 05:53:29,172 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-11 05:53:29,345 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047326686:84 , srv1072751751:84 , srv1131324612:84 , srv1148554285:84 , srv116832473:84 , srv1184488345:84 , srv1195117773:84 , srv1200474963:84 , srv1211812904:84 , srv1212753216:84 , srv1235706835:84 , srv1273663994:84 , srv1298279058:84 , srv1301036548:84 , srv1331993291:84 , srv1359098006:84 , srv1363171297:84 , srv1370135243:84 , srv1374719446:84 , srv1396375059:84 , srv1426767327:84 , srv1430699860:84 , srv1432192735:84 , srv1435290694:84 , srv1450280489:84 , srv1503868796:84 , srv153479769:84 , srv1537960685:84 , srv1538986267:84 , srv1561904176:84 , srv1572144182:84 , srv1635375696:84 , srv1636569326:84 , srv1642105099:84 , srv1695594990:84 , srv1700450863:84 , srv1742500986:84 , srv1791383500:84 , srv1824765503:84 , srv18312078:84 , srv1870002474:84 , srv1876648476:84 , srv1881436500:84 , srv1893123951:84 , srv1893338962:84 , srv1894652867:84 , srv191806184:84 , srv1940520919:84 , srv1940845480:84 , srv1957043795:84 , srv1982367490:84 , srv1995677312:84 , srv2056117471:84 , srv2076115979:84 , srv2095779469:84 , srv2106278421:84 , srv2129482604:84 , srv234694347:84 , srv238435217:84 , srv245864957:84 , srv25022110:84 , srv261425175:84 , srv261878887:84 , srv26203337:84 , srv320232446:84 , srv320899351:84 , srv347894335:84 , srv360429764:84 , srv388585234:84 , srv414895278:84 , srv425520587:84 , srv449239187:84 , srv476888415:84 , srv480471409:84 , srv485718989:84 , srv485881513:84 , srv533218720:84 , srv555858142:84 , srv588328492:84 , srv591797527:84 , srv621680688:84 , srv645764229:84 , srv699159617:84 , srv716622422:84 , srv718864226:84 , srv722789678:84 , srv749700429:84 , srv75336187:84 , srv755724066:84 , srv760611294:84 , srv792060157:84 , srv81215028:84 , srv820521309:84 , srv894573606:84 , srv917097237:84 , srv960824004:84 , srv970373193:84 , srv993493374:84 , srv998813694:84 , srv1756079400:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-11 05:53:29,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:29,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1432192735=22, srv261878887=63, srv718864226=85, srv485718989=75, srv960824004=96, srv1642105099=33, srv449239187=72, srv1695594990=34, srv347894335=67, srv245864957=60, srv480471409=74, srv621680688=81, srv1503868796=25, srv1211812904=8, srv116832473=4, srv1298279058=12, srv1940520919=48, srv1824765503=39, srv1742500986=36, srv191806184=47, srv533218720=77, srv588328492=79, srv749700429=87, srv820521309=93, srv993493374=98, srv591797527=80, srv1200474963=7, srv1363171297=16, srv722789678=86, srv2076115979=54, srv1370135243=17, srv360429764=68, srv425520587=71, srv1572144182=30, srv75336187=88, srv1538986267=28, srv760611294=90, srv1131324612=2, srv1700450863=35, srv1072751751=1, srv1047326686=0, srv1148554285=3, srv153479769=26, srv1982367490=51, srv1331993291=14, srv1396375059=19, srv1995677312=52, srv755724066=89, srv81215028=92, srv1756079400=37, srv320899351=66, srv1894652867=46, srv1893338962=45, srv261425175=62, srv1301036548=13, srv1195117773=6, srv1870002474=41, srv699159617=83, srv1430699860=21, srv18312078=40, srv716622422=84, srv1957043795=50, srv1537960685=27, srv1359098006=15, srv1450280489=24, srv234694347=58, srv320232446=65, srv917097237=95, srv388585234=69, srv1561904176=29, srv792060157=91, srv26203337=64, srv894573606=94, srv1635375696=31, srv485881513=76, srv998813694=99, srv1636569326=32, srv2129482604=57, srv1791383500=38, srv414895278=70, srv1940845480=49, srv1273663994=11, srv1893123951=44, srv645764229=82, srv476888415=73, srv2056117471=53, srv1435290694=23, srv1426767327=20, srv2095779469=55, srv1235706835=10, srv1876648476=42, srv1212753216=9, srv1184488345=5, srv1374719446=18, srv2106278421=56, srv1881436500=43, srv555858142=78, srv970373193=97, srv238435217=59, srv25022110=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:29,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:29,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:29,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:29,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:29,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:29,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:29,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:29,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:29,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-11 05:53:29,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-11 05:53:29,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-11 05:53:29,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-11 05:53:29,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-11 05:53:29,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-11 05:53:29,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-11 05:53:29,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-11 05:53:29,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-11 05:53:29,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-11 05:53:29,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-11 05:53:29,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:29,737 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-11 05:53:29,738 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-11 05:53:30,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 566 ms to try 5000 different iterations.  Found a solution that moves 4665 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17962866896702565. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04138516957655736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05535519425369401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5183333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04138516957655736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:30,092 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1538986267:82 , srv1148554285:83 , srv1331993291:83 , srv153479769:83 , srv1572144182:83 , srv191806184:83 , srv234694347:83 , srv261878887:83 , srv26203337:83 , srv1298279058:84 , srv1537960685:84 , srv1636569326:84 , srv1695594990:84 , srv1870002474:84 , srv1881436500:84 , srv1894652867:84 , srv1940845480:84 , srv2106278421:84 , srv238435217:84 , srv449239187:84 , srv555858142:84 , srv699159617:84 , srv716622422:84 , srv792060157:84 , srv993493374:84 , srv1072751751:85 , srv116832473:85 , srv1184488345:85 , srv1195117773:85 , srv1363171297:85 , srv1374719446:85 , srv1435290694:85 , srv1791383500:85 , srv1876648476:85 , srv1893123951:85 , srv1940520919:85 , srv261425175:85 , srv388585234:85 , srv414895278:85 , srv588328492:85 , srv755724066:85 , srv894573606:85 , srv1047326686:86 , srv1131324612:86 , srv1200474963:86 , srv1211812904:86 , srv1273663994:86 , srv1359098006:86 , srv1396375059:86 , srv1426767327:86 , srv18312078:86 , srv1957043795:86 , srv2076115979:86 , srv2095779469:86 , srv320899351:86 , srv533218720:86 , srv917097237:86 , srv1370135243:87 , srv1432192735:87 , srv1503868796:87 , srv1635375696:87 , srv1700450863:87 , srv1742500986:87 , srv1982367490:87 , srv1995677312:87 , srv2129482604:87 , srv480471409:87 , srv485718989:87 , srv485881513:87 , srv591797527:87 , srv645764229:87 , srv722789678:87 , srv81215028:87 , srv960824004:87 , srv1212753216:88 , srv1235706835:88 , srv1450280489:88 , srv1824765503:88 , srv25022110:88 , srv476888415:88 , srv820521309:88 , srv970373193:88 , srv1430699860:89 , srv1561904176:89 , srv1642105099:89 , srv320232446:89 , srv347894335:89 , srv75336187:89 , srv760611294:89 , srv998813694:89 , srv1893338962:90 , srv245864957:90 , srv749700429:90 , srv2056117471:91 , srv425520587:91 , srv621680688:91 , srv1301036548:93 , srv360429764:93 , srv718864226:93 , srv1756079400:458 }
2023-10-11 05:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1432192735=22, srv261878887=63, srv718864226=85, srv485718989=75, srv960824004=96, srv1642105099=33, srv449239187=72, srv1695594990=34, srv347894335=67, srv245864957=60, srv480471409=74, srv621680688=81, srv1503868796=25, srv1211812904=8, srv116832473=4, srv1298279058=12, srv1940520919=48, srv1824765503=39, srv1742500986=36, srv191806184=47, srv533218720=77, srv588328492=79, srv749700429=87, srv820521309=93, srv993493374=98, srv591797527=80, srv1200474963=7, srv1363171297=16, srv722789678=86, srv2076115979=54, srv1370135243=17, srv360429764=68, srv425520587=71, srv1572144182=30, srv75336187=88, srv1538986267=28, srv760611294=90, srv1131324612=2, srv1700450863=35, srv1072751751=1, srv1047326686=0, srv1148554285=3, srv153479769=26, srv1982367490=51, srv1331993291=14, srv1396375059=19, srv1995677312=52, srv755724066=89, srv81215028=92, srv1756079400=37, srv320899351=66, srv1894652867=46, srv1893338962=45, srv261425175=62, srv1301036548=13, srv1195117773=6, srv1870002474=41, srv699159617=83, srv1430699860=21, srv18312078=40, srv716622422=84, srv1957043795=50, srv1537960685=27, srv1359098006=15, srv1450280489=24, srv234694347=58, srv320232446=65, srv917097237=95, srv388585234=69, srv1561904176=29, srv792060157=91, srv26203337=64, srv894573606=94, srv1635375696=31, srv485881513=76, srv998813694=99, srv1636569326=32, srv2129482604=57, srv1791383500=38, srv414895278=70, srv1940845480=49, srv1273663994=11, srv1893123951=44, srv645764229=82, srv476888415=73, srv2056117471=53, srv1435290694=23, srv1426767327=20, srv2095779469=55, srv1235706835=10, srv1876648476=42, srv1212753216=9, srv1184488345=5, srv1374719446=18, srv2106278421=56, srv1881436500=43, srv555858142=78, srv970373193=97, srv238435217=59, srv25022110=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:30,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:30,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:30,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:30,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:30,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-11 05:53:30,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-11 05:53:30,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-11 05:53:30,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-11 05:53:30,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-11 05:53:30,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-11 05:53:30,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-11 05:53:30,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-11 05:53:30,282 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04138516957655736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05535519425369401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04138516957655736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:30,282 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-11 05:53:30,283 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.179596003938862, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04138516957655736, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05535519425369401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04138516957655736, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17441666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-11 05:53:30,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 427 ms to try 5000 different iterations.  Found a solution that moves 2679 regions; Going from a computed imbalance of 0.179596003938862 to a new imbalance of 0.015965371924364955. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020251566627465925); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032601391527316684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2976666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020251566627465925); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:30,608 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1301036548:78 , srv718864226:78 , srv360429764:79 , srv2056117471:80 , srv621680688:80 , srv245864957:82 , srv347894335:82 , srv1370135243:83 , srv1430699860:83 , srv1561904176:83 , srv1824765503:83 , srv1893338962:83 , srv1982367490:83 , srv425520587:83 , srv476888415:83 , srv760611294:83 , srv970373193:83 , srv998813694:83 , srv1211812904:84 , srv1235706835:84 , srv1995677312:84 , srv25022110:84 , srv320232446:84 , srv480471409:84 , srv722789678:84 , srv749700429:84 , srv75336187:84 , srv1131324612:85 , srv1212753216:85 , srv1359098006:85 , srv1450280489:85 , srv1503868796:85 , srv1635375696:85 , srv1642105099:85 , srv1700450863:85 , srv18312078:85 , srv1876648476:85 , srv2076115979:85 , srv2095779469:85 , srv2129482604:85 , srv388585234:85 , srv533218720:85 , srv591797527:85 , srv81215028:85 , srv820521309:85 , srv1047326686:86 , srv1200474963:86 , srv1363171297:86 , srv1396375059:86 , srv1426767327:86 , srv1742500986:86 , srv1957043795:86 , srv485881513:86 , srv116832473:87 , srv1195117773:87 , srv1273663994:87 , srv1374719446:87 , srv1432192735:87 , srv1695594990:87 , srv1791383500:87 , srv1870002474:87 , srv1893123951:87 , srv1940520919:87 , srv1940845480:87 , srv238435217:87 , srv588328492:87 , srv645764229:87 , srv716622422:87 , srv792060157:87 , srv894573606:87 , srv960824004:87 , srv993493374:87 , srv1072751751:88 , srv1148554285:88 , srv1184488345:88 , srv1298279058:88 , srv1435290694:88 , srv1572144182:88 , srv261425175:88 , srv26203337:88 , srv414895278:88 , srv555858142:88 , srv917097237:88 , srv153479769:89 , srv1881436500:89 , srv1894652867:89 , srv191806184:89 , srv261878887:89 , srv320899351:89 , srv1331993291:90 , srv1537960685:90 , srv1636569326:90 , srv2106278421:90 , srv449239187:90 , srv485718989:90 , srv699159617:90 , srv755724066:90 , srv234694347:91 , srv1538986267:92 , srv1756079400:496 }
2023-10-11 05:53:30,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:30,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1432192735=22, srv261878887=63, srv718864226=85, srv485718989=75, srv960824004=96, srv1642105099=33, srv449239187=72, srv1695594990=34, srv347894335=67, srv245864957=60, srv480471409=74, srv621680688=81, srv1503868796=25, srv1211812904=8, srv116832473=4, srv1298279058=12, srv1940520919=48, srv1824765503=39, srv1742500986=36, srv191806184=47, srv533218720=77, srv588328492=79, srv749700429=87, srv820521309=93, srv993493374=98, srv591797527=80, srv1200474963=7, srv1363171297=16, srv722789678=86, srv2076115979=54, srv1370135243=17, srv360429764=68, srv425520587=71, srv1572144182=30, srv75336187=88, srv1538986267=28, srv760611294=90, srv1131324612=2, srv1700450863=35, srv1072751751=1, srv1047326686=0, srv1148554285=3, srv153479769=26, srv1982367490=51, srv1331993291=14, srv1396375059=19, srv1995677312=52, srv755724066=89, srv81215028=92, srv1756079400=37, srv320899351=66, srv1894652867=46, srv1893338962=45, srv261425175=62, srv1301036548=13, srv1195117773=6, srv1870002474=41, srv699159617=83, srv1430699860=21, srv18312078=40, srv716622422=84, srv1957043795=50, srv1537960685=27, srv1359098006=15, srv1450280489=24, srv234694347=58, srv320232446=65, srv917097237=95, srv388585234=69, srv1561904176=29, srv792060157=91, srv26203337=64, srv894573606=94, srv1635375696=31, srv485881513=76, srv998813694=99, srv1636569326=32, srv2129482604=57, srv1791383500=38, srv414895278=70, srv1940845480=49, srv1273663994=11, srv1893123951=44, srv645764229=82, srv476888415=73, srv2056117471=53, srv1435290694=23, srv1426767327=20, srv2095779469=55, srv1235706835=10, srv1876648476=42, srv1212753216=9, srv1184488345=5, srv1374719446=18, srv2106278421=56, srv1881436500=43, srv555858142=78, srv970373193=97, srv238435217=59, srv25022110=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:30,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:30,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:30,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-11 05:53:30,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-11 05:53:30,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-11 05:53:30,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-11 05:53:30,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-11 05:53:30,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-11 05:53:30,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-11 05:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-11 05:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-11 05:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-11 05:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-11 05:53:30,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020251566627465925); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032601391527316684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020251566627465925); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:30,782 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-11 05:53:30,782 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01594661316542596, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020251566627465925); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032601391527316684, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020251566627465925); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-11 05:53:31,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 1012 regions; Going from a computed imbalance of 0.01594661316542596 to a new imbalance of 8.043204798690016E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.11244444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:31,128 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv485718989:80 , srv320899351:82 , srv755724066:82 , srv1537960685:83 , srv1636569326:83 , srv1642105099:83 , srv2106278421:83 , srv234694347:83 , srv449239187:83 , srv645764229:83 , srv1072751751:84 , srv1184488345:84 , srv1273663994:84 , srv1432192735:84 , srv1538986267:84 , srv1893338962:84 , srv1894652867:84 , srv425520587:84 , srv485881513:84 , srv699159617:84 , srv749700429:84 , srv75336187:84 , srv917097237:84 , srv960824004:84 , srv116832473:85 , srv1195117773:85 , srv1200474963:85 , srv1212753216:85 , srv1298279058:85 , srv1331993291:85 , srv1396375059:85 , srv1426767327:85 , srv1430699860:85 , srv1435290694:85 , srv1503868796:85 , srv153479769:85 , srv1561904176:85 , srv1635375696:85 , srv1700450863:85 , srv1742500986:85 , srv1791383500:85 , srv1881436500:85 , srv1940520919:85 , srv2129482604:85 , srv25022110:85 , srv261425175:85 , srv261878887:85 , srv320232446:85 , srv414895278:85 , srv820521309:85 , srv894573606:85 , srv1047326686:86 , srv1148554285:86 , srv1235706835:86 , srv1301036548:86 , srv1359098006:86 , srv1363171297:86 , srv1374719446:86 , srv1450280489:86 , srv1572144182:86 , srv1695594990:86 , srv18312078:86 , srv1870002474:86 , srv1876648476:86 , srv1893123951:86 , srv191806184:86 , srv1940845480:86 , srv1957043795:86 , srv2076115979:86 , srv2095779469:86 , srv238435217:86 , srv245864957:86 , srv26203337:86 , srv347894335:86 , srv360429764:86 , srv476888415:86 , srv555858142:86 , srv588328492:86 , srv591797527:86 , srv621680688:86 , srv716622422:86 , srv760611294:86 , srv792060157:86 , srv993493374:86 , srv998813694:86 , srv1131324612:87 , srv1370135243:87 , srv1824765503:87 , srv1982367490:87 , srv1995677312:87 , srv2056117471:87 , srv388585234:87 , srv480471409:87 , srv718864226:87 , srv722789678:87 , srv81215028:87 , srv1211812904:88 , srv533218720:88 , srv970373193:88 , srv1756079400:559 }
2023-10-11 05:53:31,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:31,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1432192735=22, srv261878887=63, srv718864226=85, srv485718989=75, srv960824004=96, srv1642105099=33, srv449239187=72, srv1695594990=34, srv347894335=67, srv245864957=60, srv480471409=74, srv621680688=81, srv1503868796=25, srv1211812904=8, srv116832473=4, srv1298279058=12, srv1940520919=48, srv1824765503=39, srv1742500986=36, srv191806184=47, srv533218720=77, srv588328492=79, srv749700429=87, srv820521309=93, srv993493374=98, srv591797527=80, srv1200474963=7, srv1363171297=16, srv722789678=86, srv2076115979=54, srv1370135243=17, srv360429764=68, srv425520587=71, srv1572144182=30, srv75336187=88, srv1538986267=28, srv760611294=90, srv1131324612=2, srv1700450863=35, srv1072751751=1, srv1047326686=0, srv1148554285=3, srv153479769=26, srv1982367490=51, srv1331993291=14, srv1396375059=19, srv1995677312=52, srv755724066=89, srv81215028=92, srv1756079400=37, srv320899351=66, srv1894652867=46, srv1893338962=45, srv261425175=62, srv1301036548=13, srv1195117773=6, srv1870002474=41, srv699159617=83, srv1430699860=21, srv18312078=40, srv716622422=84, srv1957043795=50, srv1537960685=27, srv1359098006=15, srv1450280489=24, srv234694347=58, srv320232446=65, srv917097237=95, srv388585234=69, srv1561904176=29, srv792060157=91, srv26203337=64, srv894573606=94, srv1635375696=31, srv485881513=76, srv998813694=99, srv1636569326=32, srv2129482604=57, srv1791383500=38, srv414895278=70, srv1940845480=49, srv1273663994=11, srv1893123951=44, srv645764229=82, srv476888415=73, srv2056117471=53, srv1435290694=23, srv1426767327=20, srv2095779469=55, srv1235706835=10, srv1876648476=42, srv1212753216=9, srv1184488345=5, srv1374719446=18, srv2106278421=56, srv1881436500=43, srv555858142=78, srv970373193=97, srv238435217=59, srv25022110=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:31,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:31,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-11 05:53:31,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-11 05:53:31,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-11 05:53:31,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-11 05:53:31,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-11 05:53:31,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-11 05:53:31,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-11 05:53:31,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-11 05:53:31,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-11 05:53:31,297 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:31,297 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-11 05:53:31,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.972343044131366E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-11 05:53:31,656 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 477 ms to try 5000 different iterations.  Found a solution that moves 248 regions; Going from a computed imbalance of 7.972343044131366E-4 to a new imbalance of 1.1728712400398278E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.027555555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:31,657 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1450280489:83 , srv1538986267:83 , srv533218720:83 , srv81215028:83 , srv970373193:83 , srv1047326686:84 , srv1131324612:84 , srv1211812904:84 , srv1212753216:84 , srv1235706835:84 , srv1301036548:84 , srv1331993291:84 , srv1370135243:84 , srv1374719446:84 , srv1430699860:84 , srv1432192735:84 , srv1435290694:84 , srv1503868796:84 , srv1742500986:84 , srv1824765503:84 , srv1881436500:84 , srv1893123951:84 , srv191806184:84 , srv1957043795:84 , srv1995677312:84 , srv2056117471:84 , srv245864957:84 , srv320232446:84 , srv347894335:84 , srv360429764:84 , srv414895278:84 , srv425520587:84 , srv480471409:84 , srv555858142:84 , srv588328492:84 , srv591797527:84 , srv699159617:84 , srv718864226:84 , srv722789678:84 , srv749700429:84 , srv760611294:84 , srv820521309:84 , srv917097237:84 , srv960824004:84 , srv1072751751:85 , srv1148554285:85 , srv116832473:85 , srv1200474963:85 , srv1273663994:85 , srv1298279058:85 , srv1359098006:85 , srv1363171297:85 , srv1396375059:85 , srv1426767327:85 , srv153479769:85 , srv1537960685:85 , srv1561904176:85 , srv1572144182:85 , srv1635375696:85 , srv1636569326:85 , srv1642105099:85 , srv1695594990:85 , srv1791383500:85 , srv18312078:85 , srv1870002474:85 , srv1893338962:85 , srv1894652867:85 , srv1940520919:85 , srv1940845480:85 , srv1982367490:85 , srv2076115979:85 , srv2095779469:85 , srv2106278421:85 , srv2129482604:85 , srv234694347:85 , srv238435217:85 , srv25022110:85 , srv261425175:85 , srv261878887:85 , srv26203337:85 , srv320899351:85 , srv388585234:85 , srv449239187:85 , srv476888415:85 , srv485718989:85 , srv485881513:85 , srv621680688:85 , srv645764229:85 , srv716622422:85 , srv75336187:85 , srv755724066:85 , srv792060157:85 , srv894573606:85 , srv993493374:85 , srv998813694:85 , srv1184488345:86 , srv1195117773:86 , srv1876648476:86 , srv1700450863:87 , srv1756079400:629 }
2023-10-11 05:53:31,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:31,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1432192735=22, srv261878887=63, srv718864226=85, srv485718989=75, srv960824004=96, srv1642105099=33, srv449239187=72, srv1695594990=34, srv347894335=67, srv245864957=60, srv480471409=74, srv621680688=81, srv1503868796=25, srv1211812904=8, srv116832473=4, srv1298279058=12, srv1940520919=48, srv1824765503=39, srv1742500986=36, srv191806184=47, srv533218720=77, srv588328492=79, srv749700429=87, srv820521309=93, srv993493374=98, srv591797527=80, srv1200474963=7, srv1363171297=16, srv722789678=86, srv2076115979=54, srv1370135243=17, srv360429764=68, srv425520587=71, srv1572144182=30, srv75336187=88, srv1538986267=28, srv760611294=90, srv1131324612=2, srv1700450863=35, srv1072751751=1, srv1047326686=0, srv1148554285=3, srv153479769=26, srv1982367490=51, srv1331993291=14, srv1396375059=19, srv1995677312=52, srv755724066=89, srv81215028=92, srv1756079400=37, srv320899351=66, srv1894652867=46, srv1893338962=45, srv261425175=62, srv1301036548=13, srv1195117773=6, srv1870002474=41, srv699159617=83, srv1430699860=21, srv18312078=40, srv716622422=84, srv1957043795=50, srv1537960685=27, srv1359098006=15, srv1450280489=24, srv234694347=58, srv320232446=65, srv917097237=95, srv388585234=69, srv1561904176=29, srv792060157=91, srv26203337=64, srv894573606=94, srv1635375696=31, srv485881513=76, srv998813694=99, srv1636569326=32, srv2129482604=57, srv1791383500=38, srv414895278=70, srv1940845480=49, srv1273663994=11, srv1893123951=44, srv645764229=82, srv476888415=73, srv2056117471=53, srv1435290694=23, srv1426767327=20, srv2095779469=55, srv1235706835=10, srv1876648476=42, srv1212753216=9, srv1184488345=5, srv1374719446=18, srv2106278421=56, srv1881436500=43, srv555858142=78, srv970373193=97, srv238435217=59, srv25022110=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:31,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:31,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:31,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-11 05:53:31,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-11 05:53:31,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-11 05:53:31,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-11 05:53:31,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-11 05:53:31,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-11 05:53:31,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-11 05:53:31,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-11 05:53:31,829 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.992179284731768E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:31,829 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1450280489:83 , srv1538986267:83 , srv533218720:83 , srv81215028:83 , srv970373193:83 , srv1047326686:84 , srv1131324612:84 , srv1211812904:84 , srv1212753216:84 , srv1235706835:84 , srv1301036548:84 , srv1331993291:84 , srv1370135243:84 , srv1374719446:84 , srv1430699860:84 , srv1432192735:84 , srv1435290694:84 , srv1503868796:84 , srv1742500986:84 , srv1824765503:84 , srv1881436500:84 , srv1893123951:84 , srv191806184:84 , srv1957043795:84 , srv1995677312:84 , srv2056117471:84 , srv245864957:84 , srv320232446:84 , srv347894335:84 , srv360429764:84 , srv414895278:84 , srv425520587:84 , srv480471409:84 , srv555858142:84 , srv588328492:84 , srv591797527:84 , srv699159617:84 , srv718864226:84 , srv722789678:84 , srv749700429:84 , srv760611294:84 , srv820521309:84 , srv917097237:84 , srv960824004:84 , srv1072751751:85 , srv1148554285:85 , srv116832473:85 , srv1200474963:85 , srv1273663994:85 , srv1298279058:85 , srv1359098006:85 , srv1363171297:85 , srv1396375059:85 , srv1426767327:85 , srv153479769:85 , srv1537960685:85 , srv1561904176:85 , srv1572144182:85 , srv1635375696:85 , srv1636569326:85 , srv1642105099:85 , srv1695594990:85 , srv1791383500:85 , srv18312078:85 , srv1870002474:85 , srv1893338962:85 , srv1894652867:85 , srv1940520919:85 , srv1940845480:85 , srv1982367490:85 , srv2076115979:85 , srv2095779469:85 , srv2106278421:85 , srv2129482604:85 , srv234694347:85 , srv238435217:85 , srv25022110:85 , srv261425175:85 , srv261878887:85 , srv26203337:85 , srv320899351:85 , srv388585234:85 , srv449239187:85 , srv476888415:85 , srv485718989:85 , srv485881513:85 , srv621680688:85 , srv645764229:85 , srv716622422:85 , srv75336187:85 , srv755724066:85 , srv792060157:85 , srv894573606:85 , srv993493374:85 , srv998813694:85 , srv1184488345:86 , srv1195117773:86 , srv1876648476:86 , srv1700450863:87 , srv1756079400:629 }
2023-10-11 05:53:31,865 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=310 (was 276) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=7130 (was 7799)
2023-10-11 05:53:31,877 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=171, AvailableMemoryMB=7130
2023-10-11 05:53:31,877 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-11 05:53:31,878 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-11 05:53:31,890 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1062802511:84 , srv1106394673:84 , srv1140456028:84 , srv1173548059:84 , srv1220456866:84 , srv1234601930:84 , srv1240282047:84 , srv138876061:84 , srv1498440601:84 , srv1525242020:84 , srv1619495878:84 , srv1685713644:84 , srv1719931769:84 , srv1796521234:84 , srv1850424650:84 , srv1988734834:84 , srv1997212953:84 , srv2009761112:84 , srv2120884483:84 , srv27644853:84 , srv304469100:84 , srv395298265:84 , srv397789918:84 , srv45841701:84 , srv477666352:84 , srv489745842:84 , srv605086936:84 , srv661024606:84 , srv731193459:84 , srv1138741235:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-11 05:53:31,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:53:31,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1997212953=17, srv1498440601=9, srv395298265=22, srv1138741235=2, srv1850424650=15, srv731193459=29, srv27644853=20, srv304469100=21, srv2120884483=19, srv1525242020=10, srv1173548059=4, srv1719931769=13, srv477666352=25, srv661024606=28, srv1140456028=3, srv489745842=26, srv1685713644=12, srv397789918=23, srv1062802511=0, srv2009761112=18, srv1220456866=5, srv1988734834=16, srv1240282047=7, srv138876061=8, srv1106394673=1, srv1234601930=6, srv1619495878=11, srv1796521234=14, srv605086936=27, srv45841701=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:53:31,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:53:31,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:53:31,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:53:31,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:53:31,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-11 05:53:31,932 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.069943284144976, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:53:31,932 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912413432268763, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.069943284144976, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-11 05:55:31,910 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6065061 different iterations.  Found a solution that moves 1847 regions; Going from a computed imbalance of 0.9912413432268763 to a new imbalance of 4.3109901406398434E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6840740740740741, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:55:31,915 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1062802511:90 , srv1106394673:90 , srv1138741235:90 , srv1140456028:90 , srv1173548059:90 , srv1220456866:90 , srv1234601930:90 , srv1240282047:90 , srv138876061:90 , srv1498440601:90 , srv1525242020:90 , srv1619495878:90 , srv1685713644:90 , srv1719931769:90 , srv1796521234:90 , srv1850424650:90 , srv1988734834:90 , srv1997212953:90 , srv2009761112:90 , srv2120884483:90 , srv27644853:90 , srv304469100:90 , srv395298265:90 , srv397789918:90 , srv45841701:90 , srv477666352:90 , srv489745842:90 , srv605086936:90 , srv661024606:90 , srv731193459:90 }
2023-10-11 05:55:31,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-11 05:55:31,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1997212953=17, srv1498440601=9, srv395298265=22, srv1138741235=2, srv1850424650=15, srv731193459=29, srv27644853=20, srv304469100=21, srv2120884483=19, srv1525242020=10, srv1173548059=4, srv1719931769=13, srv477666352=25, srv661024606=28, srv1140456028=3, srv489745842=26, srv1685713644=12, srv397789918=23, srv1062802511=0, srv2009761112=18, srv1220456866=5, srv1988734834=16, srv1240282047=7, srv138876061=8, srv1106394673=1, srv1234601930=6, srv1619495878=11, srv1796521234=14, srv605086936=27, srv45841701=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-11 05:55:31,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-11 05:55:31,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-11 05:55:31,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-11 05:55:31,933 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1062802511:90 , srv1106394673:90 , srv1138741235:90 , srv1140456028:90 , srv1173548059:90 , srv1220456866:90 , srv1234601930:90 , srv1240282047:90 , srv138876061:90 , srv1498440601:90 , srv1525242020:90 , srv1619495878:90 , srv1685713644:90 , srv1719931769:90 , srv1796521234:90 , srv1850424650:90 , srv1988734834:90 , srv1997212953:90 , srv2009761112:90 , srv2120884483:90 , srv27644853:90 , srv304469100:90 , srv395298265:90 , srv397789918:90 , srv45841701:90 , srv477666352:90 , srv489745842:90 , srv605086936:90 , srv661024606:90 , srv731193459:90 }
2023-10-11 05:55:31,948 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=242 (was 310), ProcessCount=167 (was 171), AvailableMemoryMB=6293 (was 7130)