2023-11-01 13:56:48,180 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-01 13:56:48,419 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-01 13:56:48,508 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-01 13:56:48,564 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 13:56:48,570 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-01 13:56:48,580 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=167, AvailableMemoryMB=6145
2023-11-01 13:56:48,582 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 13:56:48,583 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-01 13:56:48,700 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001960048:84 , srv100356712:84 , srv1012005785:84 , srv1021959369:84 , srv1022032494:84 , srv103850863:84 , srv1072709269:84 , srv1075739887:84 , srv1089551338:84 , srv1096663154:84 , srv1147462340:84 , srv1190209646:84 , srv1213620673:84 , srv1233128001:84 , srv1283383266:84 , srv1284988853:84 , srv1289770969:84 , srv1292324443:84 , srv1328853753:84 , srv1335135466:84 , srv1361083999:84 , srv1367301153:84 , srv1376759141:84 , srv1409639947:84 , srv1416161205:84 , srv1416912760:84 , srv1424023033:84 , srv1431250895:84 , srv1468732083:84 , srv1476557091:84 , srv149895250:84 , srv1521454540:84 , srv1523408680:84 , srv1530933599:84 , srv1531836928:84 , srv1541180381:84 , srv1551017226:84 , srv1568145941:84 , srv1588322483:84 , srv16080994:84 , srv1630075021:84 , srv1631760659:84 , srv1642147502:84 , srv1662403007:84 , srv1698466281:84 , srv1703144600:84 , srv1741800140:84 , srv1792375923:84 , srv1844210179:84 , srv1854582005:84 , srv1860561070:84 , srv1903504245:84 , srv1954026784:84 , srv1966106174:84 , srv1980426360:84 , srv2013677470:84 , srv2037240578:84 , srv2059048657:84 , srv2066105357:84 , srv2093407908:84 , srv209424664:84 , srv2131027016:84 , srv2139805313:84 , srv243203525:84 , srv246545555:84 , srv293108733:84 , srv373747768:84 , srv377070906:84 , srv401607777:84 , srv412264606:84 , srv42235500:84 , srv435525814:84 , srv441395398:84 , srv459927910:84 , srv462538577:84 , srv46378302:84 , srv476922204:84 , srv479977285:84 , srv527783564:84 , srv555023632:84 , srv582555983:84 , srv585241971:84 , srv60938555:84 , srv614883161:84 , srv625931561:84 , srv653181809:84 , srv658926609:84 , srv659254906:84 , srv699141846:84 , srv715186812:84 , srv830389712:84 , srv85623945:84 , srv86003457:84 , srv897308657:84 , srv900012381:84 , srv926754108:84 , srv959724946:84 , srv996431042:84 , srv999477000:84 , srv890547867:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-01 13:56:48,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:48,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:48,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:48,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:48,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:48,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:48,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:48,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:48,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:48,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:48,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:48,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:48,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:48,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:48,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:48,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:48,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:48,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:48,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:48,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:48,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:48,940 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:48,940 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 13:56:48,940 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 13:56:49,216 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 397 ms to try 5000 different iterations.  Found a solution that moves 4687 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18162438092715957. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04141318331266976, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5207777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04141318331266976, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17691666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,237 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1416161205:81 , srv1416912760:81 , srv830389712:81 , srv1468732083:82 , srv1588322483:82 , srv614883161:82 , srv1001960048:83 , srv1213620673:83 , srv1289770969:83 , srv1476557091:83 , srv1642147502:83 , srv1703144600:83 , srv2037240578:83 , srv2131027016:83 , srv246545555:83 , srv46378302:83 , srv699141846:83 , srv85623945:83 , srv1233128001:84 , srv1521454540:84 , srv1662403007:84 , srv1954026784:84 , srv209424664:84 , srv2139805313:84 , srv373747768:84 , srv527783564:84 , srv1021959369:85 , srv149895250:85 , srv1531836928:85 , srv1698466281:85 , srv1844210179:85 , srv2059048657:85 , srv459927910:85 , srv476922204:85 , srv60938555:85 , srv658926609:85 , srv86003457:85 , srv1075739887:86 , srv1147462340:86 , srv1283383266:86 , srv1284988853:86 , srv1376759141:86 , srv1431250895:86 , srv1523408680:86 , srv1530933599:86 , srv1630075021:86 , srv1860561070:86 , srv2013677470:86 , srv2066105357:86 , srv42235500:86 , srv555023632:86 , srv582555983:86 , srv585241971:86 , srv897308657:86 , srv103850863:87 , srv1072709269:87 , srv1292324443:87 , srv1361083999:87 , srv1367301153:87 , srv2093407908:87 , srv243203525:87 , srv377070906:87 , srv401607777:87 , srv435525814:87 , srv462538577:87 , srv715186812:87 , srv900012381:87 , srv926754108:87 , srv1012005785:88 , srv1022032494:88 , srv1089551338:88 , srv1328853753:88 , srv1335135466:88 , srv1409639947:88 , srv1424023033:88 , srv1966106174:88 , srv412264606:88 , srv659254906:88 , srv959724946:88 , srv996431042:88 , srv1096663154:89 , srv1190209646:89 , srv1541180381:89 , srv1568145941:89 , srv16080994:89 , srv1631760659:89 , srv1741800140:89 , srv1854582005:89 , srv1903504245:89 , srv625931561:89 , srv1792375923:90 , srv1980426360:90 , srv479977285:90 , srv653181809:90 , srv441395398:91 , srv999477000:91 , srv1551017226:92 , srv293108733:93 , srv100356712:97 , srv890547867:458 }
2023-11-01 13:56:49,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:49,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:49,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:49,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:49,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:49,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:49,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:49,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:49,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:49,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:49,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:49,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:49,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:49,389 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04141318331266976, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04141318331266976, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17691666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,390 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 13:56:49,390 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18159156185170341, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04141318331266976, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05697972545876103, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04141318331266976, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17691666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 13:56:49,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 254 ms to try 5000 different iterations.  Found a solution that moves 2688 regions; Going from a computed imbalance of 0.18159156185170341 to a new imbalance of 0.013971965856796043. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020373737095518617); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03128020425846319, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2986666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020373737095518617); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,566 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv100356712:75 , srv293108733:78 , srv1551017226:79 , srv999477000:80 , srv653181809:81 , srv1096663154:82 , srv1541180381:82 , srv1631760659:82 , srv1741800140:82 , srv1792375923:82 , srv1854582005:82 , srv479977285:82 , srv1022032494:83 , srv1089551338:83 , srv1190209646:83 , srv1367301153:83 , srv1568145941:83 , srv1966106174:83 , srv1980426360:83 , srv2093407908:83 , srv441395398:83 , srv625931561:83 , srv959724946:83 , srv996431042:83 , srv1072709269:84 , srv1292324443:84 , srv377070906:84 , srv659254906:84 , srv926754108:84 , srv103850863:85 , srv1075739887:85 , srv1283383266:85 , srv1335135466:85 , srv1376759141:85 , srv16080994:85 , srv1903504245:85 , srv2013677470:85 , srv401607777:85 , srv42235500:85 , srv476922204:85 , srv582555983:85 , srv897308657:85 , srv900012381:85 , srv1012005785:86 , srv1424023033:86 , srv1431250895:86 , srv1530933599:86 , srv1630075021:86 , srv1844210179:86 , srv2059048657:86 , srv243203525:86 , srv412264606:86 , srv459927910:86 , srv462538577:86 , srv715186812:86 , srv1021959369:87 , srv1233128001:87 , srv1328853753:87 , srv1361083999:87 , srv1409639947:87 , srv149895250:87 , srv1662403007:87 , srv1954026784:87 , srv2139805313:87 , srv46378302:87 , srv527783564:87 , srv60938555:87 , srv658926609:87 , srv1001960048:88 , srv1147462340:88 , srv1213620673:88 , srv1289770969:88 , srv1523408680:88 , srv1698466281:88 , srv1860561070:88 , srv2037240578:88 , srv2066105357:88 , srv209424664:88 , srv435525814:88 , srv86003457:88 , srv1476557091:89 , srv1531836928:89 , srv1642147502:89 , srv373747768:89 , srv555023632:89 , srv585241971:89 , srv85623945:89 , srv1284988853:90 , srv1416912760:90 , srv1521454540:90 , srv2131027016:90 , srv246545555:90 , srv699141846:90 , srv830389712:90 , srv1416161205:91 , srv1588322483:91 , srv1703144600:91 , srv1468732083:92 , srv614883161:93 , srv890547867:497 }
2023-11-01 13:56:49,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:49,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:49,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:49,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:49,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:49,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:49,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:49,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020373737095518617); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03128020425846319, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020373737095518617); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,687 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 13:56:49,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013953144078510112, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020373737095518617); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03128020425846319, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020373737095518617); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 13:56:49,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 229 ms to try 5000 different iterations.  Found a solution that moves 987 regions; Going from a computed imbalance of 0.013953144078510112 to a new imbalance of 0.0010593802653726498. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005085624249954194); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008368612192811304); MoveCostFunction : (multiplier=7.0, imbalance=0.10966666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005085624249954194); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,844 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1284988853:82 , srv1328853753:82 , srv555023632:82 , srv585241971:82 , srv1147462340:83 , srv1409639947:83 , srv1424023033:83 , srv1468732083:83 , srv1521454540:83 , srv1531836928:83 , srv16080994:83 , srv1703144600:83 , srv1903504245:83 , srv435525814:83 , srv441395398:83 , srv614883161:83 , srv1012005785:84 , srv1335135466:84 , srv1361083999:84 , srv1523408680:84 , srv1588322483:84 , srv1860561070:84 , srv2066105357:84 , srv2131027016:84 , srv246545555:84 , srv412264606:84 , srv699141846:84 , srv86003457:84 , srv100356712:85 , srv103850863:85 , srv149895250:85 , srv1530933599:85 , srv1630075021:85 , srv1642147502:85 , srv1698466281:85 , srv1792375923:85 , srv1980426360:85 , srv243203525:85 , srv373747768:85 , srv401607777:85 , srv462538577:85 , srv479977285:85 , srv60938555:85 , srv658926609:85 , srv715186812:85 , srv900012381:85 , srv1021959369:86 , srv1096663154:86 , srv1190209646:86 , srv1283383266:86 , srv1289770969:86 , srv1292324443:86 , srv1416161205:86 , srv1416912760:86 , srv1431250895:86 , srv1476557091:86 , srv1568145941:86 , srv1631760659:86 , srv1662403007:86 , srv1844210179:86 , srv1854582005:86 , srv1954026784:86 , srv1966106174:86 , srv2013677470:86 , srv2059048657:86 , srv209424664:86 , srv2139805313:86 , srv527783564:86 , srv582555983:86 , srv625931561:86 , srv659254906:86 , srv830389712:86 , srv85623945:86 , srv1001960048:87 , srv1022032494:87 , srv1072709269:87 , srv1075739887:87 , srv1089551338:87 , srv1213620673:87 , srv1233128001:87 , srv1541180381:87 , srv1551017226:87 , srv1741800140:87 , srv2037240578:87 , srv293108733:87 , srv377070906:87 , srv42235500:87 , srv459927910:87 , srv653181809:87 , srv897308657:87 , srv926754108:87 , srv959724946:87 , srv996431042:87 , srv999477000:87 , srv1367301153:88 , srv1376759141:88 , srv2093407908:88 , srv46378302:88 , srv476922204:89 , srv890547867:548 }
2023-11-01 13:56:49,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:49,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:49,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:49,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:49,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:49,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:49,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:49,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:49,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:49,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:49,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:49,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:49,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:49,942 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005085624249954194); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008368612192811304); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005085624249954194); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:49,942 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 13:56:49,942 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0010524691436582835, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005085624249954194); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008368612192811304); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005085624249954194); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 13:56:50,159 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 289 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 0.0010524691436582835 to a new imbalance of 3.7693965842610886E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024158102511315956); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:50,160 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1233128001:83 , srv1367301153:83 , srv1376759141:83 , srv1476557091:83 , srv715186812:83 , srv1001960048:84 , srv100356712:84 , srv1021959369:84 , srv1022032494:84 , srv1072709269:84 , srv1075739887:84 , srv1089551338:84 , srv1213620673:84 , srv1284988853:84 , srv1361083999:84 , srv1416161205:84 , srv1416912760:84 , srv1431250895:84 , srv1523408680:84 , srv1530933599:84 , srv1541180381:84 , srv1551017226:84 , srv1568145941:84 , srv1698466281:84 , srv1741800140:84 , srv1854582005:84 , srv1860561070:84 , srv1980426360:84 , srv2037240578:84 , srv2066105357:84 , srv209424664:84 , srv243203525:84 , srv373747768:84 , srv377070906:84 , srv412264606:84 , srv42235500:84 , srv435525814:84 , srv459927910:84 , srv462538577:84 , srv46378302:84 , srv476922204:84 , srv582555983:84 , srv614883161:84 , srv625931561:84 , srv653181809:84 , srv659254906:84 , srv85623945:84 , srv897308657:84 , srv926754108:84 , srv959724946:84 , srv996431042:84 , srv999477000:84 , srv1012005785:85 , srv103850863:85 , srv1096663154:85 , srv1147462340:85 , srv1190209646:85 , srv1283383266:85 , srv1289770969:85 , srv1292324443:85 , srv1328853753:85 , srv1409639947:85 , srv1424023033:85 , srv1468732083:85 , srv149895250:85 , srv1521454540:85 , srv1531836928:85 , srv16080994:85 , srv1630075021:85 , srv1631760659:85 , srv1642147502:85 , srv1662403007:85 , srv1703144600:85 , srv1792375923:85 , srv1844210179:85 , srv1903504245:85 , srv1954026784:85 , srv1966106174:85 , srv2013677470:85 , srv2059048657:85 , srv2093407908:85 , srv2131027016:85 , srv2139805313:85 , srv246545555:85 , srv293108733:85 , srv401607777:85 , srv441395398:85 , srv479977285:85 , srv527783564:85 , srv585241971:85 , srv60938555:85 , srv658926609:85 , srv699141846:85 , srv830389712:85 , srv86003457:85 , srv900012381:85 , srv1335135466:86 , srv1588322483:86 , srv555023632:86 , srv890547867:639 }
2023-11-01 13:56:50,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:50,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:50,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:50,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:50,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:50,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:50,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:50,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:50,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:50,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:50,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:50,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:50,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:50,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024158102511315956); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:50,259 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-01 13:56:50,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.566334244122667E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024158102511315956); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-01 13:56:50,408 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 222 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 3.566334244122667E-5 to a new imbalance of 8.761718501804472E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:50,408 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1012005785:83 , srv1190209646:83 , srv1335135466:83 , srv1409639947:83 , srv1588322483:83 , srv2093407908:83 , srv293108733:83 , srv476922204:83 , srv555023632:83 , srv1001960048:84 , srv1021959369:84 , srv1022032494:84 , srv103850863:84 , srv1072709269:84 , srv1075739887:84 , srv1089551338:84 , srv1096663154:84 , srv1147462340:84 , srv1213620673:84 , srv1283383266:84 , srv1284988853:84 , srv1289770969:84 , srv1292324443:84 , srv1328853753:84 , srv1361083999:84 , srv1376759141:84 , srv1416161205:84 , srv1424023033:84 , srv1431250895:84 , srv1468732083:84 , srv149895250:84 , srv1521454540:84 , srv1523408680:84 , srv1531836928:84 , srv1541180381:84 , srv1551017226:84 , srv1568145941:84 , srv16080994:84 , srv1630075021:84 , srv1631760659:84 , srv1642147502:84 , srv1662403007:84 , srv1698466281:84 , srv1703144600:84 , srv1741800140:84 , srv1792375923:84 , srv1844210179:84 , srv1860561070:84 , srv1903504245:84 , srv1954026784:84 , srv1966106174:84 , srv1980426360:84 , srv2013677470:84 , srv2037240578:84 , srv2059048657:84 , srv2066105357:84 , srv209424664:84 , srv2131027016:84 , srv2139805313:84 , srv243203525:84 , srv246545555:84 , srv373747768:84 , srv377070906:84 , srv401607777:84 , srv412264606:84 , srv42235500:84 , srv435525814:84 , srv441395398:84 , srv459927910:84 , srv462538577:84 , srv46378302:84 , srv479977285:84 , srv527783564:84 , srv585241971:84 , srv60938555:84 , srv614883161:84 , srv625931561:84 , srv653181809:84 , srv658926609:84 , srv659254906:84 , srv699141846:84 , srv830389712:84 , srv85623945:84 , srv86003457:84 , srv897308657:84 , srv900012381:84 , srv926754108:84 , srv959724946:84 , srv996431042:84 , srv999477000:84 , srv100356712:85 , srv1233128001:85 , srv1367301153:85 , srv1416912760:85 , srv1476557091:85 , srv1530933599:85 , srv1854582005:85 , srv582555983:85 , srv715186812:85 , srv890547867:684 }
2023-11-01 13:56:50,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:50,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2013677470=55, srv1089551338=8, srv1703144600=45, srv441395398=72, srv412264606=69, srv1147462340=10, srv614883161=83, srv435525814=71, srv100356712=1, srv103850863=5, srv1523408680=32, srv999477000=99, srv1190209646=11, srv462538577=74, srv1012005785=2, srv699141846=88, srv243203525=63, srv149895250=30, srv1367301153=21, srv585241971=81, srv1844210179=48, srv1292324443=17, srv401607777=68, srv86003457=92, srv996431042=98, srv1424023033=26, srv900012381=95, srv1903504245=51, srv1289770969=16, srv209424664=60, srv1233128001=13, srv1021959369=3, srv890547867=93, srv1001960048=0, srv1361083999=20, srv1698466281=44, srv653181809=85, srv1328853753=18, srv1409639947=23, srv1860561070=50, srv1568145941=37, srv897308657=94, srv2059048657=57, srv2066105357=58, srv555023632=79, srv1741800140=46, srv1631760659=41, srv1283383266=14, srv1630075021=40, srv1642147502=42, srv1662403007=43, srv60938555=82, srv1521454540=31, srv1431250895=27, srv527783564=78, srv1376759141=22, srv715186812=89, srv1966106174=53, srv582555983=80, srv2131027016=61, srv1416912760=25, srv1792375923=47, srv625931561=84, srv2139805313=62, srv377070906=67, srv926754108=96, srv85623945=91, srv959724946=97, srv1980426360=54, srv42235500=70, srv459927910=73, srv1468732083=28, srv373747768=66, srv1541180381=35, srv293108733=65, srv1854582005=49, srv1072709269=6, srv1284988853=15, srv1551017226=36, srv830389712=90, srv1530933599=33, srv1213620673=12, srv479977285=77, srv1096663154=9, srv1335135466=19, srv658926609=86, srv1416161205=24, srv2037240578=56, srv476922204=76, srv46378302=75, srv246545555=64, srv1588322483=38, srv1954026784=52, srv1022032494=4, srv16080994=39, srv659254906=87, srv1476557091=29, srv1075739887=7, srv2093407908=59, srv1531836928=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:50,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:50,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-01 13:56:50,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-01 13:56:50,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-01 13:56:50,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-01 13:56:50,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-01 13:56:50,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-01 13:56:50,510 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.530647563026267E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:50,510 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1012005785:83 , srv1190209646:83 , srv1335135466:83 , srv1409639947:83 , srv1588322483:83 , srv2093407908:83 , srv293108733:83 , srv476922204:83 , srv555023632:83 , srv1001960048:84 , srv1021959369:84 , srv1022032494:84 , srv103850863:84 , srv1072709269:84 , srv1075739887:84 , srv1089551338:84 , srv1096663154:84 , srv1147462340:84 , srv1213620673:84 , srv1283383266:84 , srv1284988853:84 , srv1289770969:84 , srv1292324443:84 , srv1328853753:84 , srv1361083999:84 , srv1376759141:84 , srv1416161205:84 , srv1424023033:84 , srv1431250895:84 , srv1468732083:84 , srv149895250:84 , srv1521454540:84 , srv1523408680:84 , srv1531836928:84 , srv1541180381:84 , srv1551017226:84 , srv1568145941:84 , srv16080994:84 , srv1630075021:84 , srv1631760659:84 , srv1642147502:84 , srv1662403007:84 , srv1698466281:84 , srv1703144600:84 , srv1741800140:84 , srv1792375923:84 , srv1844210179:84 , srv1860561070:84 , srv1903504245:84 , srv1954026784:84 , srv1966106174:84 , srv1980426360:84 , srv2013677470:84 , srv2037240578:84 , srv2059048657:84 , srv2066105357:84 , srv209424664:84 , srv2131027016:84 , srv2139805313:84 , srv243203525:84 , srv246545555:84 , srv373747768:84 , srv377070906:84 , srv401607777:84 , srv412264606:84 , srv42235500:84 , srv435525814:84 , srv441395398:84 , srv459927910:84 , srv462538577:84 , srv46378302:84 , srv479977285:84 , srv527783564:84 , srv585241971:84 , srv60938555:84 , srv614883161:84 , srv625931561:84 , srv653181809:84 , srv658926609:84 , srv659254906:84 , srv699141846:84 , srv830389712:84 , srv85623945:84 , srv86003457:84 , srv897308657:84 , srv900012381:84 , srv926754108:84 , srv959724946:84 , srv996431042:84 , srv999477000:84 , srv100356712:85 , srv1233128001:85 , srv1367301153:85 , srv1416912760:85 , srv1476557091:85 , srv1530933599:85 , srv1854582005:85 , srv582555983:85 , srv715186812:85 , srv890547867:684 }
2023-11-01 13:56:50,531 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=291 (was 291), ProcessCount=167 (was 167), AvailableMemoryMB=5647 (was 6145)
2023-11-01 13:56:50,537 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=167, AvailableMemoryMB=5647
2023-11-01 13:56:50,538 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-01 13:56:50,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-01 13:56:50,548 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1075597526:84 , srv1081373438:84 , srv1122517349:84 , srv1176460640:84 , srv1187121377:84 , srv1187336815:84 , srv1230222906:84 , srv1374759724:84 , srv1428546591:84 , srv1763302834:84 , srv1788669683:84 , srv1845028922:84 , srv1865693246:84 , srv194223061:84 , srv1952081656:84 , srv2042938620:84 , srv2066458723:84 , srv2075730197:84 , srv284919009:84 , srv340279120:84 , srv440600151:84 , srv440723281:84 , srv486890294:84 , srv565045369:84 , srv731924102:84 , srv781221908:84 , srv800947405:84 , srv810616150:84 , srv971719975:84 , srv1172099255:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-01 13:56:50,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:56:50,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv971719975=29, srv1428546591=9, srv1075597526=0, srv1081373438=1, srv1788669683=11, srv800947405=27, srv440723281=22, srv1187121377=5, srv340279120=20, srv1952081656=15, srv2075730197=18, srv1763302834=10, srv440600151=21, srv1374759724=8, srv194223061=14, srv2066458723=17, srv731924102=25, srv1122517349=2, srv1187336815=6, srv565045369=24, srv1845028922=12, srv1865693246=13, srv486890294=23, srv2042938620=16, srv781221908=26, srv1172099255=3, srv1176460640=4, srv284919009=19, srv810616150=28, srv1230222906=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:56:50,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:56:50,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:56:50,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:56:50,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:56:50,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:56:50,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:56:50,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:56:50,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:56:50,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-01 13:56:50,591 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1500134036669056, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:56:50,591 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912665730600846, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1500134036669056, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-01 13:58:50,570 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9281469 different iterations.  Found a solution that moves 1843 regions; Going from a computed imbalance of 0.9912665730600846 to a new imbalance of 4.3016539410932495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6825925925925926, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:58:50,576 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1075597526:90 , srv1081373438:90 , srv1122517349:90 , srv1172099255:90 , srv1176460640:90 , srv1187121377:90 , srv1187336815:90 , srv1230222906:90 , srv1374759724:90 , srv1428546591:90 , srv1763302834:90 , srv1788669683:90 , srv1845028922:90 , srv1865693246:90 , srv194223061:90 , srv1952081656:90 , srv2042938620:90 , srv2066458723:90 , srv2075730197:90 , srv284919009:90 , srv340279120:90 , srv440600151:90 , srv440723281:90 , srv486890294:90 , srv565045369:90 , srv731924102:90 , srv781221908:90 , srv800947405:90 , srv810616150:90 , srv971719975:90 }
2023-11-01 13:58:50,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-01 13:58:50,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv971719975=29, srv1428546591=9, srv1075597526=0, srv1081373438=1, srv1788669683=11, srv800947405=27, srv440723281=22, srv1187121377=5, srv340279120=20, srv1952081656=15, srv2075730197=18, srv1763302834=10, srv440600151=21, srv1374759724=8, srv194223061=14, srv2066458723=17, srv731924102=25, srv1122517349=2, srv1187336815=6, srv565045369=24, srv1845028922=12, srv1865693246=13, srv486890294=23, srv2042938620=16, srv781221908=26, srv1172099255=3, srv1176460640=4, srv284919009=19, srv810616150=28, srv1230222906=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-01 13:58:50,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-01 13:58:50,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-01 13:58:50,592 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-01 13:58:50,592 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1075597526:90 , srv1081373438:90 , srv1122517349:90 , srv1172099255:90 , srv1176460640:90 , srv1187121377:90 , srv1187336815:90 , srv1230222906:90 , srv1374759724:90 , srv1428546591:90 , srv1763302834:90 , srv1788669683:90 , srv1845028922:90 , srv1865693246:90 , srv194223061:90 , srv1952081656:90 , srv2042938620:90 , srv2066458723:90 , srv2075730197:90 , srv284919009:90 , srv340279120:90 , srv440600151:90 , srv440723281:90 , srv486890294:90 , srv565045369:90 , srv731924102:90 , srv781221908:90 , srv800947405:90 , srv810616150:90 , srv971719975:90 }
2023-11-01 13:58:50,603 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=268 (was 291), ProcessCount=166 (was 167), AvailableMemoryMB=5953 (was 5647) - AvailableMemoryMB LEAK? -