2024-04-15 05:55:22,570 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-15 05:55:22,810 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-15 05:55:22,901 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-15 05:55:22,960 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 05:55:22,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-15 05:55:22,984 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=216, ProcessCount=179, AvailableMemoryMB=2607
2024-04-15 05:55:22,986 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 05:55:22,986 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-15 05:55:23,114 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000137875:84 , srv1039444049:84 , srv1052646225:84 , srv1067013913:84 , srv1099979870:84 , srv1105267002:84 , srv1233914053:84 , srv1239776278:84 , srv1314723302:84 , srv1319766864:84 , srv1338483265:84 , srv1374044810:84 , srv1380146857:84 , srv1401641005:84 , srv1411944773:84 , srv1414730785:84 , srv1437825041:84 , srv1437915867:84 , srv1441109837:84 , srv1446643824:84 , srv1453353147:84 , srv146209380:84 , srv1470879220:84 , srv1487712547:84 , srv1495804701:84 , srv1506085379:84 , srv1514111714:84 , srv151517099:84 , srv1572629229:84 , srv1573219531:84 , srv1576982623:84 , srv1582175169:84 , srv1582430799:84 , srv1594891158:84 , srv1595437578:84 , srv1602266692:84 , srv1607687091:84 , srv1620283787:84 , srv1625564429:84 , srv1638544586:84 , srv1702998556:84 , srv171733461:84 , srv1740111333:84 , srv1744668839:84 , srv1744858620:84 , srv1752189253:84 , srv175556189:84 , srv1760631885:84 , srv1777847900:84 , srv1849517254:84 , srv1858251974:84 , srv1869393939:84 , srv1875727306:84 , srv1936517267:84 , srv1947656849:84 , srv1957062244:84 , srv1988199772:84 , srv2016023147:84 , srv2071987397:84 , srv2102322403:84 , srv2116499991:84 , srv2117695330:84 , srv280143060:84 , srv283700869:84 , srv291824587:84 , srv300125363:84 , srv341962131:84 , srv358540247:84 , srv385681283:84 , srv422222794:84 , srv428758528:84 , srv429722724:84 , srv432382358:84 , srv447858162:84 , srv450072061:84 , srv489780616:84 , srv497107288:84 , srv515824261:84 , srv548536980:84 , srv557429667:84 , srv560545439:84 , srv61689517:84 , srv62039989:84 , srv621929345:84 , srv626142540:84 , srv638409010:84 , srv649563249:84 , srv676953368:84 , srv690088766:84 , srv698007194:84 , srv716042226:84 , srv728925278:84 , srv782912501:84 , srv811877516:84 , srv821152159:84 , srv834140460:84 , srv852733466:84 , srv910035437:84 , srv926356492:84 , srv1990089190:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-15 05:55:23,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:23,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:23,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:23,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:23,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:23,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:23,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:23,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:23,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:23,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:23,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:23,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:23,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:23,442 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:23,443 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 05:55:23,443 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 05:55:23,703 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 456 ms to try 5000 different iterations.  Found a solution that moves 4701 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17677106870570844. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039944156054960156, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05534708360247777, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5223333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039944156054960156, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:23,713 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1869393939:81 , srv2071987397:81 , srv811877516:81 , srv1233914053:82 , srv2102322403:82 , srv638409010:82 , srv1411944773:83 , srv1414730785:83 , srv1582175169:83 , srv1607687091:83 , srv698007194:83 , srv821152159:83 , srv1000137875:84 , srv1401641005:84 , srv1446643824:84 , srv1453353147:84 , srv151517099:84 , srv1572629229:84 , srv1602266692:84 , srv1625564429:84 , srv557429667:84 , srv560545439:84 , srv728925278:84 , srv910035437:84 , srv1067013913:85 , srv1437915867:85 , srv1595437578:85 , srv1620283787:85 , srv1760631885:85 , srv1957062244:85 , srv2116499991:85 , srv2117695330:85 , srv432382358:85 , srv447858162:85 , srv489780616:85 , srv649563249:85 , srv782912501:85 , srv926356492:85 , srv1099979870:86 , srv1338483265:86 , srv1441109837:86 , srv1514111714:86 , srv1573219531:86 , srv1638544586:86 , srv2016023147:86 , srv300125363:86 , srv429722724:86 , srv497107288:86 , srv1314723302:87 , srv1576982623:87 , srv1582430799:87 , srv1702998556:87 , srv1744668839:87 , srv175556189:87 , srv1849517254:87 , srv1858251974:87 , srv1875727306:87 , srv1988199772:87 , srv280143060:87 , srv358540247:87 , srv548536980:87 , srv62039989:87 , srv621929345:87 , srv626142540:87 , srv676953368:87 , srv716042226:87 , srv834140460:87 , srv1374044810:88 , srv1380146857:88 , srv1437825041:88 , srv1470879220:88 , srv1487712547:88 , srv1506085379:88 , srv171733461:88 , srv1752189253:88 , srv428758528:88 , srv450072061:88 , srv515824261:88 , srv690088766:88 , srv852733466:88 , srv1239776278:89 , srv1594891158:89 , srv1740111333:89 , srv1936517267:89 , srv283700869:89 , srv341962131:89 , srv61689517:89 , srv1052646225:90 , srv1319766864:90 , srv146209380:90 , srv1744858620:90 , srv1947656849:90 , srv422222794:90 , srv1039444049:91 , srv1105267002:91 , srv1495804701:91 , srv1777847900:92 , srv385681283:92 , srv291824587:93 , srv1990089190:445 }
2024-04-15 05:55:23,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:23,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:23,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:23,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:23,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:23,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:23,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:23,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:23,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:23,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:23,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:23,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:23,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:23,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:23,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:23,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039944156054960156, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05534708360247777, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039944156054960156, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:23,866 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 05:55:23,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17673815160015705, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039944156054960156, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05534708360247777, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039944156054960156, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17183333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 05:55:24,048 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 272 ms to try 5000 different iterations.  Found a solution that moves 2656 regions; Going from a computed imbalance of 0.17673815160015705 to a new imbalance of 0.01621317734050892. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02027249212356149); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030999076775651897, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2951111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02027249212356149); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048916666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,056 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv291824587:78 , srv1105267002:79 , srv1039444049:80 , srv1740111333:81 , srv1947656849:81 , srv385681283:81 , srv515824261:81 , srv146209380:82 , srv1777847900:82 , srv422222794:82 , srv1319766864:83 , srv1380146857:83 , srv1487712547:83 , srv1744858620:83 , srv1752189253:83 , srv1936517267:83 , srv428758528:83 , srv450072061:83 , srv690088766:83 , srv852733466:83 , srv1052646225:84 , srv1239776278:84 , srv1374044810:84 , srv1594891158:84 , srv171733461:84 , srv1744668839:84 , srv280143060:84 , srv300125363:84 , srv1099979870:85 , srv1314723302:85 , srv1338483265:85 , srv1495804701:85 , srv1573219531:85 , srv1576982623:85 , srv1607687091:85 , srv1638544586:85 , srv1875727306:85 , srv1988199772:85 , srv2016023147:85 , srv283700869:85 , srv341962131:85 , srv432382358:85 , srv548536980:85 , srv62039989:85 , srv626142540:85 , srv834140460:85 , srv1437825041:86 , srv1437915867:86 , srv1441109837:86 , srv1446643824:86 , srv1470879220:86 , srv1506085379:86 , srv151517099:86 , srv1702998556:86 , srv2117695330:86 , srv358540247:86 , srv489780616:86 , srv497107288:86 , srv560545439:86 , srv621929345:86 , srv782912501:86 , srv1233914053:87 , srv1401641005:87 , srv1514111714:87 , srv1582430799:87 , srv1595437578:87 , srv175556189:87 , srv1760631885:87 , srv1858251974:87 , srv61689517:87 , srv716042226:87 , srv728925278:87 , srv910035437:87 , srv926356492:87 , srv1000137875:88 , srv1453353147:88 , srv1620283787:88 , srv1849517254:88 , srv1957062244:88 , srv2116499991:88 , srv429722724:88 , srv649563249:88 , srv676953368:88 , srv821152159:88 , srv1067013913:89 , srv1572629229:89 , srv1602266692:89 , srv1625564429:89 , srv638409010:89 , srv1411944773:90 , srv1414730785:90 , srv1582175169:90 , srv447858162:90 , srv557429667:90 , srv1869393939:91 , srv2071987397:91 , srv2102322403:91 , srv811877516:91 , srv698007194:92 , srv1990089190:509 }
2024-04-15 05:55:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:24,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:24,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:24,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:24,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:24,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:24,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:24,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:24,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:24,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:24,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:24,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:24,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:24,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:24,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:24,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02027249212356149); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030999076775651897, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02027249212356149); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048916666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,185 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 05:55:24,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0161945796310121, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02027249212356149); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030999076775651897, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02027249212356149); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.048916666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 05:55:24,341 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 235 ms to try 5000 different iterations.  Found a solution that moves 1021 regions; Going from a computed imbalance of 0.0161945796310121 to a new imbalance of 4.376991790387952E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009125635134638571); MoveCostFunction : (multiplier=7.0, imbalance=0.11344444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,343 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1495804701:82 , srv61689517:82 , srv676953368:82 , srv698007194:82 , srv1052646225:83 , srv1067013913:83 , srv1437825041:83 , srv1849517254:83 , srv1858251974:83 , srv283700869:83 , srv447858162:83 , srv557429667:83 , srv716042226:83 , srv1239776278:84 , srv1319766864:84 , srv1411944773:84 , srv1414730785:84 , srv1470879220:84 , srv1506085379:84 , srv1582175169:84 , srv1582430799:84 , srv1594891158:84 , srv1602266692:84 , srv1625564429:84 , srv175556189:84 , srv1777847900:84 , srv2116499991:84 , srv341962131:84 , srv429722724:84 , srv1314723302:85 , srv1441109837:85 , srv1514111714:85 , srv1572629229:85 , srv1595437578:85 , srv1620283787:85 , srv1702998556:85 , srv1744858620:85 , srv1869393939:85 , srv1957062244:85 , srv2102322403:85 , srv358540247:85 , srv385681283:85 , srv422222794:85 , srv497107288:85 , srv626142540:85 , srv649563249:85 , srv690088766:85 , srv834140460:85 , srv1000137875:86 , srv1039444049:86 , srv1374044810:86 , srv1380146857:86 , srv1401641005:86 , srv1437915867:86 , srv1453353147:86 , srv146209380:86 , srv1487712547:86 , srv1573219531:86 , srv1576982623:86 , srv171733461:86 , srv1744668839:86 , srv1752189253:86 , srv1760631885:86 , srv1936517267:86 , srv1947656849:86 , srv2016023147:86 , srv2071987397:86 , srv2117695330:86 , srv280143060:86 , srv291824587:86 , srv428758528:86 , srv548536980:86 , srv560545439:86 , srv62039989:86 , srv621929345:86 , srv638409010:86 , srv728925278:86 , srv811877516:86 , srv821152159:86 , srv852733466:86 , srv910035437:86 , srv926356492:86 , srv1099979870:87 , srv1105267002:87 , srv1338483265:87 , srv1446643824:87 , srv1638544586:87 , srv1740111333:87 , srv1875727306:87 , srv1988199772:87 , srv300125363:87 , srv432382358:87 , srv450072061:87 , srv489780616:87 , srv782912501:87 , srv1233914053:88 , srv151517099:88 , srv515824261:88 , srv1607687091:89 , srv1990089190:558 }
2024-04-15 05:55:24,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:24,347 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:24,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:24,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:24,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:24,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:24,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:24,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:24,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:24,439 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009125635134638571); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,439 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 05:55:24,439 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.305499842359907E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060960430498352425); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009125635134638571); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060960430498352425); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 05:55:24,662 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 293 ms to try 5000 different iterations.  Found a solution that moves 256 regions; Going from a computed imbalance of 4.305499842359907E-4 to a new imbalance of 3.6000902225948114E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.028444444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,662 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv146209380:83 , srv171733461:83 , srv1875727306:83 , srv1957062244:83 , srv1988199772:83 , srv1000137875:84 , srv1099979870:84 , srv1338483265:84 , srv1374044810:84 , srv1470879220:84 , srv1495804701:84 , srv1506085379:84 , srv1514111714:84 , srv151517099:84 , srv1572629229:84 , srv1576982623:84 , srv1582430799:84 , srv1594891158:84 , srv1602266692:84 , srv1620283787:84 , srv1638544586:84 , srv1702998556:84 , srv1740111333:84 , srv1744858620:84 , srv175556189:84 , srv1760631885:84 , srv1777847900:84 , srv1849517254:84 , srv1936517267:84 , srv2071987397:84 , srv341962131:84 , srv358540247:84 , srv385681283:84 , srv429722724:84 , srv432382358:84 , srv447858162:84 , srv450072061:84 , srv489780616:84 , srv61689517:84 , srv62039989:84 , srv621929345:84 , srv626142540:84 , srv649563249:84 , srv782912501:84 , srv811877516:84 , srv910035437:84 , srv926356492:84 , srv1039444049:85 , srv1052646225:85 , srv1067013913:85 , srv1105267002:85 , srv1233914053:85 , srv1239776278:85 , srv1319766864:85 , srv1380146857:85 , srv1401641005:85 , srv1411944773:85 , srv1414730785:85 , srv1437825041:85 , srv1437915867:85 , srv1441109837:85 , srv1446643824:85 , srv1453353147:85 , srv1487712547:85 , srv1573219531:85 , srv1582175169:85 , srv1607687091:85 , srv1625564429:85 , srv1744668839:85 , srv1752189253:85 , srv1858251974:85 , srv1869393939:85 , srv1947656849:85 , srv2016023147:85 , srv2102322403:85 , srv2117695330:85 , srv280143060:85 , srv283700869:85 , srv291824587:85 , srv300125363:85 , srv422222794:85 , srv497107288:85 , srv515824261:85 , srv548536980:85 , srv557429667:85 , srv638409010:85 , srv690088766:85 , srv698007194:85 , srv716042226:85 , srv728925278:85 , srv821152159:85 , srv834140460:85 , srv852733466:85 , srv1314723302:86 , srv1595437578:86 , srv2116499991:86 , srv428758528:86 , srv560545439:86 , srv676953368:86 , srv1990089190:631 }
2024-04-15 05:55:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:24,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:24,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:24,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:24,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:24,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:24,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:24,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:24,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:24,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:24,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:24,759 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,759 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 05:55:24,759 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.420835191300204E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 05:55:24,912 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 223 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 3.420835191300204E-5 to a new imbalance of 7.878479841169313E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:24,912 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1314723302:83 , srv1453353147:83 , srv1595437578:83 , srv2102322403:83 , srv2116499991:83 , srv428758528:83 , srv548536980:83 , srv621929345:83 , srv676953368:83 , srv1000137875:84 , srv1039444049:84 , srv1052646225:84 , srv1067013913:84 , srv1099979870:84 , srv1105267002:84 , srv1233914053:84 , srv1239776278:84 , srv1319766864:84 , srv1338483265:84 , srv1374044810:84 , srv1380146857:84 , srv1401641005:84 , srv1411944773:84 , srv1414730785:84 , srv1437825041:84 , srv1437915867:84 , srv1441109837:84 , srv1446643824:84 , srv1470879220:84 , srv1487712547:84 , srv1495804701:84 , srv1506085379:84 , srv1514111714:84 , srv151517099:84 , srv1572629229:84 , srv1573219531:84 , srv1576982623:84 , srv1582175169:84 , srv1582430799:84 , srv1607687091:84 , srv1620283787:84 , srv1625564429:84 , srv1638544586:84 , srv1702998556:84 , srv1744668839:84 , srv1744858620:84 , srv1752189253:84 , srv175556189:84 , srv1760631885:84 , srv1777847900:84 , srv1849517254:84 , srv1858251974:84 , srv1869393939:84 , srv1875727306:84 , srv1936517267:84 , srv1947656849:84 , srv1988199772:84 , srv2016023147:84 , srv2071987397:84 , srv2117695330:84 , srv280143060:84 , srv283700869:84 , srv291824587:84 , srv300125363:84 , srv341962131:84 , srv358540247:84 , srv385681283:84 , srv422222794:84 , srv429722724:84 , srv447858162:84 , srv450072061:84 , srv489780616:84 , srv497107288:84 , srv515824261:84 , srv557429667:84 , srv560545439:84 , srv61689517:84 , srv62039989:84 , srv638409010:84 , srv649563249:84 , srv698007194:84 , srv716042226:84 , srv728925278:84 , srv782912501:84 , srv811877516:84 , srv821152159:84 , srv834140460:84 , srv852733466:84 , srv926356492:84 , srv146209380:85 , srv1594891158:85 , srv1602266692:85 , srv171733461:85 , srv1740111333:85 , srv1957062244:85 , srv432382358:85 , srv626142540:85 , srv690088766:85 , srv910035437:85 , srv1990089190:683 }
2024-04-15 05:55:24,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:24,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1437825041=16, srv1052646225=2, srv1401641005=13, srv560545439=81, srv1374044810=11, srv557429667=80, srv175556189=46, srv1067013913=3, srv626142540=85, srv1411944773=14, srv811877516=94, srv62039989=83, srv291824587=65, srv638409010=86, srv728925278=92, srv341962131=67, srv910035437=98, srv1777847900=48, srv146209380=21, srv2071987397=59, srv2117695330=62, srv1638544586=39, srv358540247=68, srv515824261=78, srv1572629229=28, srv548536980=79, srv1988199772=56, srv1744858620=44, srv497107288=77, srv1858251974=50, srv1594891158=33, srv1446643824=19, srv422222794=70, srv1625564429=38, srv1495804701=24, srv821152159=95, srv1620283787=37, srv782912501=93, srv1487712547=23, srv1582430799=32, srv926356492=99, srv450072061=75, srv1239776278=7, srv280143060=63, srv428758528=71, srv649563249=87, srv1314723302=8, srv300125363=66, srv698007194=90, srv283700869=64, srv2116499991=61, srv1607687091=36, srv852733466=97, srv1233914053=6, srv1990089190=57, srv2102322403=60, srv447858162=74, srv1740111333=42, srv1573219531=29, srv1702998556=40, srv834140460=96, srv151517099=27, srv1602266692=35, srv429722724=72, srv1506085379=25, srv61689517=82, srv489780616=76, srv1470879220=22, srv1947656849=54, srv1039444049=1, srv385681283=69, srv1414730785=15, srv432382358=73, srv690088766=89, srv1105267002=5, srv1936517267=53, srv716042226=91, srv1595437578=34, srv1319766864=9, srv1437915867=17, srv1514111714=26, srv2016023147=58, srv1582175169=31, srv621929345=84, srv1380146857=12, srv1875727306=52, srv1338483265=10, srv1000137875=0, srv676953368=88, srv1099979870=4, srv1744668839=43, srv1752189253=45, srv1849517254=49, srv1760631885=47, srv1576982623=30, srv1453353147=20, srv171733461=41, srv1441109837=18, srv1957062244=55, srv1869393939=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 05:55:24,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 05:55:24,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 05:55:24,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 05:55:24,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 05:55:24,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 05:55:25,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.689421800350782E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:25,007 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1314723302:83 , srv1453353147:83 , srv1595437578:83 , srv2102322403:83 , srv2116499991:83 , srv428758528:83 , srv548536980:83 , srv621929345:83 , srv676953368:83 , srv1000137875:84 , srv1039444049:84 , srv1052646225:84 , srv1067013913:84 , srv1099979870:84 , srv1105267002:84 , srv1233914053:84 , srv1239776278:84 , srv1319766864:84 , srv1338483265:84 , srv1374044810:84 , srv1380146857:84 , srv1401641005:84 , srv1411944773:84 , srv1414730785:84 , srv1437825041:84 , srv1437915867:84 , srv1441109837:84 , srv1446643824:84 , srv1470879220:84 , srv1487712547:84 , srv1495804701:84 , srv1506085379:84 , srv1514111714:84 , srv151517099:84 , srv1572629229:84 , srv1573219531:84 , srv1576982623:84 , srv1582175169:84 , srv1582430799:84 , srv1607687091:84 , srv1620283787:84 , srv1625564429:84 , srv1638544586:84 , srv1702998556:84 , srv1744668839:84 , srv1744858620:84 , srv1752189253:84 , srv175556189:84 , srv1760631885:84 , srv1777847900:84 , srv1849517254:84 , srv1858251974:84 , srv1869393939:84 , srv1875727306:84 , srv1936517267:84 , srv1947656849:84 , srv1988199772:84 , srv2016023147:84 , srv2071987397:84 , srv2117695330:84 , srv280143060:84 , srv283700869:84 , srv291824587:84 , srv300125363:84 , srv341962131:84 , srv358540247:84 , srv385681283:84 , srv422222794:84 , srv429722724:84 , srv447858162:84 , srv450072061:84 , srv489780616:84 , srv497107288:84 , srv515824261:84 , srv557429667:84 , srv560545439:84 , srv61689517:84 , srv62039989:84 , srv638409010:84 , srv649563249:84 , srv698007194:84 , srv716042226:84 , srv728925278:84 , srv782912501:84 , srv811877516:84 , srv821152159:84 , srv834140460:84 , srv852733466:84 , srv926356492:84 , srv146209380:85 , srv1594891158:85 , srv1602266692:85 , srv171733461:85 , srv1740111333:85 , srv1957062244:85 , srv432382358:85 , srv626142540:85 , srv690088766:85 , srv910035437:85 , srv1990089190:683 }
2024-04-15 05:55:25,035 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=214 (was 216), ProcessCount=179 (was 179), AvailableMemoryMB=2129 (was 2607)
2024-04-15 05:55:25,042 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=214, ProcessCount=179, AvailableMemoryMB=2129
2024-04-15 05:55:25,043 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 05:55:25,043 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-15 05:55:25,053 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1012663984:84 , srv1046845293:84 , srv1078170436:84 , srv1170813431:84 , srv119314248:84 , srv1228150828:84 , srv129583345:84 , srv1344512643:84 , srv1429132366:84 , srv1468192650:84 , srv1565705921:84 , srv1577459878:84 , srv1582432291:84 , srv1593009481:84 , srv1650882281:84 , srv1665292972:84 , srv1767001523:84 , srv1838939470:84 , srv1870818737:84 , srv1898907365:84 , srv1991386100:84 , srv2082467359:84 , srv251939356:84 , srv319786157:84 , srv33943214:84 , srv518203263:84 , srv590977294:84 , srv649606831:84 , srv838324274:84 , srv1096408499:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-15 05:55:25,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:55:25,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1991386100=21, srv1870818737=19, srv1898907365=20, srv319786157=24, srv1170813431=4, srv1577459878=12, srv1565705921=11, srv1593009481=14, srv838324274=29, srv129583345=7, srv1665292972=16, srv1046845293=1, srv1012663984=0, srv590977294=27, srv1228150828=6, srv1650882281=15, srv33943214=25, srv1582432291=13, srv1429132366=9, srv1838939470=18, srv119314248=5, srv2082467359=22, srv1468192650=10, srv649606831=28, srv518203263=26, srv251939356=23, srv1344512643=8, srv1096408499=3, srv1078170436=2, srv1767001523=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:55:25,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:55:25,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:55:25,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:55:25,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:55:25,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:55:25,070 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:55:25,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-15 05:55:25,093 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1098747358672931, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:55:25,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912539254969257, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1098747358672931, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-15 05:57:25,074 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 8892655 different iterations.  Found a solution that moves 1843 regions; Going from a computed imbalance of 0.9912539254969257 to a new imbalance of 4.3016539410932495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6825925925925926, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:57:25,079 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1012663984:90 , srv1046845293:90 , srv1078170436:90 , srv1096408499:90 , srv1170813431:90 , srv119314248:90 , srv1228150828:90 , srv129583345:90 , srv1344512643:90 , srv1429132366:90 , srv1468192650:90 , srv1565705921:90 , srv1577459878:90 , srv1582432291:90 , srv1593009481:90 , srv1650882281:90 , srv1665292972:90 , srv1767001523:90 , srv1838939470:90 , srv1870818737:90 , srv1898907365:90 , srv1991386100:90 , srv2082467359:90 , srv251939356:90 , srv319786157:90 , srv33943214:90 , srv518203263:90 , srv590977294:90 , srv649606831:90 , srv838324274:90 }
2024-04-15 05:57:25,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 05:57:25,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1991386100=21, srv1870818737=19, srv1898907365=20, srv319786157=24, srv1170813431=4, srv1577459878=12, srv1565705921=11, srv1593009481=14, srv838324274=29, srv129583345=7, srv1665292972=16, srv1046845293=1, srv1012663984=0, srv590977294=27, srv1228150828=6, srv1650882281=15, srv33943214=25, srv1582432291=13, srv1429132366=9, srv1838939470=18, srv119314248=5, srv2082467359=22, srv1468192650=10, srv649606831=28, srv518203263=26, srv251939356=23, srv1344512643=8, srv1096408499=3, srv1078170436=2, srv1767001523=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 05:57:25,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 05:57:25,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 05:57:25,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 05:57:25,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 05:57:25,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-15 05:57:25,095 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 05:57:25,095 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1012663984:90 , srv1046845293:90 , srv1078170436:90 , srv1096408499:90 , srv1170813431:90 , srv119314248:90 , srv1228150828:90 , srv129583345:90 , srv1344512643:90 , srv1429132366:90 , srv1468192650:90 , srv1565705921:90 , srv1577459878:90 , srv1582432291:90 , srv1593009481:90 , srv1650882281:90 , srv1665292972:90 , srv1767001523:90 , srv1838939470:90 , srv1870818737:90 , srv1898907365:90 , srv1991386100:90 , srv2082467359:90 , srv251939356:90 , srv319786157:90 , srv33943214:90 , srv518203263:90 , srv590977294:90 , srv649606831:90 , srv838324274:90 }
2024-04-15 05:57:25,108 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=185 (was 214), ProcessCount=180 (was 179) - ProcessCount LEAK? -, AvailableMemoryMB=3134 (was 2129) - AvailableMemoryMB LEAK? -