2024-04-15 10:52:47,101 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-15 10:52:47,399 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-15 10:52:47,513 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-15 10:52:47,581 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 10:52:47,587 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-15 10:52:47,604 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=180, AvailableMemoryMB=3324
2024-04-15 10:52:47,607 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 10:52:47,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-15 10:52:47,764 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv101859221:84 , srv1038683419:84 , srv1040886835:84 , srv1097269675:84 , srv1105500464:84 , srv112585497:84 , srv1182977905:84 , srv1186802307:84 , srv1207362753:84 , srv1219664427:84 , srv1237047127:84 , srv126019790:84 , srv1297468719:84 , srv1302693158:84 , srv1302792554:84 , srv1311775070:84 , srv1328250732:84 , srv1335056124:84 , srv1362740531:84 , srv1383434323:84 , srv141738160:84 , srv1422202552:84 , srv1460308037:84 , srv1471538066:84 , srv1489807596:84 , srv15139328:84 , srv1581696043:84 , srv1640887290:84 , srv1661761053:84 , srv1666568567:84 , srv1673489380:84 , srv1727862448:84 , srv1760265860:84 , srv1761688590:84 , srv1777569441:84 , srv1791437339:84 , srv1800350395:84 , srv1813390388:84 , srv1814917860:84 , srv1876115216:84 , srv1884148106:84 , srv1891241708:84 , srv1913776312:84 , srv1991791802:84 , srv1999301138:84 , srv2036817085:84 , srv2073815508:84 , srv2075708087:84 , srv2091648978:84 , srv2098530118:84 , srv2127754805:84 , srv2131750938:84 , srv233353144:84 , srv240410061:84 , srv240765783:84 , srv257219290:84 , srv287553087:84 , srv290959015:84 , srv31377852:84 , srv332178132:84 , srv362632965:84 , srv403982525:84 , srv409831432:84 , srv411185136:84 , srv428606718:84 , srv45229090:84 , srv47004007:84 , srv480232474:84 , srv503838219:84 , srv521585099:84 , srv521973380:84 , srv547846007:84 , srv572556615:84 , srv586738416:84 , srv590706231:84 , srv596568473:84 , srv60723814:84 , srv649443716:84 , srv700082097:84 , srv700967242:84 , srv711115555:84 , srv737371694:84 , srv750932553:84 , srv76681718:84 , srv782932398:84 , srv783597004:84 , srv784796567:84 , srv832472129:84 , srv851385851:84 , srv875472968:84 , srv877167561:84 , srv877735034:84 , srv8854124:84 , srv888372852:84 , srv903983232:84 , srv935624544:84 , srv943496727:84 , srv950482829:84 , srv967091409:84 , srv1478247474:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-15 10:52:47,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:47,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:47,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:47,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:47,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:47,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:47,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:47,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:47,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:47,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:47,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:47,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:47,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:47,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:47,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:47,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:47,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:47,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:47,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:47,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:48,173 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:48,174 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 10:52:48,174 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 10:52:48,540 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 613 ms to try 5000 different iterations.  Found a solution that moves 4725 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17736166527855304. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04006510476745861, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05638372619049382, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.525, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04006510476745861, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:48,557 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv126019790:81 , srv1673489380:82 , srv1760265860:82 , srv45229090:82 , srv112585497:83 , srv1186802307:83 , srv1489807596:83 , srv1791437339:83 , srv1913776312:83 , srv1999301138:83 , srv2091648978:83 , srv2127754805:83 , srv287553087:83 , srv875472968:83 , srv1383434323:84 , srv1460308037:84 , srv15139328:84 , srv1891241708:84 , srv240410061:84 , srv240765783:84 , srv332178132:84 , srv403982525:84 , srv586738416:84 , srv649443716:84 , srv700967242:84 , srv711115555:84 , srv1038683419:85 , srv1311775070:85 , srv1581696043:85 , srv2073815508:85 , srv290959015:85 , srv31377852:85 , srv428606718:85 , srv547846007:85 , srv590706231:85 , srv851385851:85 , srv877735034:85 , srv943496727:85 , srv967091409:85 , srv101859221:86 , srv1182977905:86 , srv1302693158:86 , srv141738160:86 , srv1471538066:86 , srv1640887290:86 , srv1661761053:86 , srv1666568567:86 , srv1777569441:86 , srv1813390388:86 , srv521973380:86 , srv737371694:86 , srv76681718:86 , srv8854124:86 , srv1219664427:87 , srv1422202552:87 , srv1727862448:87 , srv1800350395:87 , srv2036817085:87 , srv2075708087:87 , srv2131750938:87 , srv257219290:87 , srv362632965:87 , srv47004007:87 , srv596568473:87 , srv60723814:87 , srv750932553:87 , srv783597004:87 , srv832472129:87 , srv877167561:87 , srv1237047127:88 , srv1297468719:88 , srv1335056124:88 , srv1362740531:88 , srv1876115216:88 , srv409831432:88 , srv480232474:88 , srv572556615:88 , srv903983232:88 , srv935624544:88 , srv1040886835:89 , srv1302792554:89 , srv1328250732:89 , srv1761688590:89 , srv1814917860:89 , srv1884148106:89 , srv1991791802:89 , srv411185136:89 , srv700082097:89 , srv1097269675:90 , srv782932398:90 , srv784796567:90 , srv950482829:90 , srv503838219:91 , srv888372852:91 , srv2098530118:92 , srv1105500464:93 , srv1207362753:93 , srv233353144:93 , srv521585099:94 , srv1478247474:446 }
2024-04-15 10:52:48,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:48,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:48,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:48,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:48,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:48,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:48,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:48,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:48,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:48,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:48,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:48,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:48,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:48,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:48,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:48,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:48,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:48,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04006510476745861, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05638372619049382, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04006510476745861, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:48,763 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 10:52:48,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1773285801214098, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04006510476745861, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05638372619049382, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04006510476745861, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 10:52:48,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 2612 regions; Going from a computed imbalance of 0.1773285801214098 to a new imbalance of 0.01690275267634813. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022170534726323774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03556186812649842, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2902222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022170534726323774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:48,957 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2098530118:78 , srv1207362753:79 , srv521585099:79 , srv888372852:79 , srv1097269675:80 , srv1105500464:80 , srv233353144:81 , srv503838219:81 , srv784796567:81 , srv1362740531:82 , srv1814917860:82 , srv1991791802:82 , srv737371694:82 , srv935624544:82 , srv950482829:82 , srv1302792554:83 , srv1328250732:83 , srv1761688590:83 , srv2075708087:83 , srv362632965:83 , srv411185136:83 , srv480232474:83 , srv782932398:83 , srv877167561:83 , srv1040886835:84 , srv1297468719:84 , srv1335056124:84 , srv1422202552:84 , srv1800350395:84 , srv1876115216:84 , srv1884148106:84 , srv2131750938:84 , srv409831432:84 , srv750932553:84 , srv783597004:84 , srv1182977905:85 , srv1237047127:85 , srv1640887290:85 , srv1666568567:85 , srv1727862448:85 , srv1777569441:85 , srv1813390388:85 , srv290959015:85 , srv572556615:85 , srv596568473:85 , srv60723814:85 , srv700082097:85 , srv8854124:85 , srv1219664427:86 , srv1302693158:86 , srv141738160:86 , srv1581696043:86 , srv2036817085:86 , srv2091648978:86 , srv31377852:86 , srv332178132:86 , srv428606718:86 , srv47004007:86 , srv547846007:86 , srv1186802307:87 , srv15139328:87 , srv1661761053:87 , srv1760265860:87 , srv257219290:87 , srv590706231:87 , srv649443716:87 , srv700967242:87 , srv832472129:87 , srv851385851:87 , srv903983232:87 , srv1038683419:88 , srv1311775070:88 , srv1891241708:88 , srv2073815508:88 , srv2127754805:88 , srv240410061:88 , srv240765783:88 , srv76681718:88 , srv875472968:88 , srv877735034:88 , srv101859221:89 , srv1383434323:89 , srv1460308037:89 , srv1471538066:89 , srv1673489380:89 , srv1999301138:89 , srv403982525:89 , srv711115555:89 , srv943496727:89 , srv967091409:89 , srv112585497:90 , srv1489807596:90 , srv521973380:90 , srv586738416:90 , srv126019790:91 , srv1791437339:91 , srv1913776312:91 , srv287553087:91 , srv45229090:92 , srv1478247474:525 }
2024-04-15 10:52:48,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:48,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:48,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:48,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:48,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:48,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:48,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:48,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:48,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:48,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:48,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:48,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:48,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:48,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:48,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:48,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:48,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:49,097 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022170534726323774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03556186812649842, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022170534726323774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,098 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 10:52:49,098 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01688446306143635, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022170534726323774); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03556186812649842, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022170534726323774); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 10:52:49,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 237 ms to try 5000 different iterations.  Found a solution that moves 972 regions; Going from a computed imbalance of 0.01688446306143635 to a new imbalance of 8.405257064927276E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006876604665976965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009522847474256706); MoveCostFunction : (multiplier=7.0, imbalance=0.108, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006876604665976965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,251 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv521973380:81 , srv101859221:82 , srv1471538066:82 , srv903983232:82 , srv1791437339:83 , srv1913776312:83 , srv233353144:83 , srv45229090:83 , srv943496727:83 , srv967091409:83 , srv1040886835:84 , srv1311775070:84 , srv1460308037:84 , srv1489807596:84 , srv1661761053:84 , srv2036817085:84 , srv2073815508:84 , srv240765783:84 , srv257219290:84 , srv287553087:84 , srv403982525:84 , srv47004007:84 , srv521585099:84 , srv586738416:84 , srv700082097:84 , srv711115555:84 , srv76681718:84 , srv832472129:84 , srv877735034:84 , srv1038683419:85 , srv1105500464:85 , srv112585497:85 , srv1207362753:85 , srv1219664427:85 , srv126019790:85 , srv1302693158:85 , srv1328250732:85 , srv1335056124:85 , srv1383434323:85 , srv141738160:85 , srv1876115216:85 , srv1884148106:85 , srv409831432:85 , srv503838219:85 , srv572556615:85 , srv590706231:85 , srv60723814:85 , srv782932398:85 , srv851385851:85 , srv950482829:85 , srv1182977905:86 , srv1237047127:86 , srv1297468719:86 , srv1302792554:86 , srv15139328:86 , srv1581696043:86 , srv1640887290:86 , srv1666568567:86 , srv1727862448:86 , srv1761688590:86 , srv1777569441:86 , srv1813390388:86 , srv1814917860:86 , srv1891241708:86 , srv1991791802:86 , srv1999301138:86 , srv2127754805:86 , srv2131750938:86 , srv290959015:86 , srv411185136:86 , srv428606718:86 , srv480232474:86 , srv596568473:86 , srv700967242:86 , srv783597004:86 , srv8854124:86 , srv1097269675:87 , srv1186802307:87 , srv1362740531:87 , srv1422202552:87 , srv1673489380:87 , srv1800350395:87 , srv2075708087:87 , srv240410061:87 , srv31377852:87 , srv332178132:87 , srv362632965:87 , srv547846007:87 , srv649443716:87 , srv750932553:87 , srv784796567:87 , srv877167561:87 , srv888372852:87 , srv2091648978:88 , srv2098530118:88 , srv875472968:88 , srv935624544:88 , srv1760265860:89 , srv737371694:90 , srv1478247474:548 }
2024-04-15 10:52:49,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:49,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:49,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:49,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:49,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:49,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:49,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:49,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:49,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:49,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:49,350 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006876604665976965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009522847474256706); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006876604665976965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,350 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 10:52:49,350 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.337196170232604E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006876604665976965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009522847474256706); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006876604665976965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 10:52:49,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 288 ms to try 5000 different iterations.  Found a solution that moves 263 regions; Going from a computed imbalance of 8.337196170232604E-4 to a new imbalance of 4.0116146058142286E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.029222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,566 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1237047127:83 , srv1727862448:83 , srv240410061:83 , srv832472129:83 , srv875472968:83 , srv1038683419:84 , srv1105500464:84 , srv1219664427:84 , srv1297468719:84 , srv1302792554:84 , srv1383434323:84 , srv1422202552:84 , srv1640887290:84 , srv1673489380:84 , srv1760265860:84 , srv1761688590:84 , srv1800350395:84 , srv1884148106:84 , srv1891241708:84 , srv1999301138:84 , srv2098530118:84 , srv257219290:84 , srv287553087:84 , srv31377852:84 , srv411185136:84 , srv480232474:84 , srv547846007:84 , srv586738416:84 , srv596568473:84 , srv649443716:84 , srv700082097:84 , srv737371694:84 , srv750932553:84 , srv76681718:84 , srv782932398:84 , srv784796567:84 , srv935624544:84 , srv101859221:85 , srv1040886835:85 , srv1097269675:85 , srv112585497:85 , srv1182977905:85 , srv1186802307:85 , srv1207362753:85 , srv126019790:85 , srv1302693158:85 , srv1311775070:85 , srv1328250732:85 , srv1335056124:85 , srv1362740531:85 , srv141738160:85 , srv1460308037:85 , srv1471538066:85 , srv1489807596:85 , srv15139328:85 , srv1581696043:85 , srv1661761053:85 , srv1666568567:85 , srv1777569441:85 , srv1791437339:85 , srv1813390388:85 , srv1876115216:85 , srv1913776312:85 , srv1991791802:85 , srv2036817085:85 , srv2073815508:85 , srv2075708087:85 , srv2091648978:85 , srv2127754805:85 , srv2131750938:85 , srv233353144:85 , srv332178132:85 , srv362632965:85 , srv403982525:85 , srv409831432:85 , srv428606718:85 , srv45229090:85 , srv47004007:85 , srv503838219:85 , srv521585099:85 , srv521973380:85 , srv572556615:85 , srv590706231:85 , srv60723814:85 , srv700967242:85 , srv711115555:85 , srv851385851:85 , srv877167561:85 , srv877735034:85 , srv8854124:85 , srv888372852:85 , srv903983232:85 , srv943496727:85 , srv950482829:85 , srv967091409:85 , srv1814917860:86 , srv240765783:86 , srv290959015:86 , srv783597004:86 , srv1478247474:623 }
2024-04-15 10:52:49,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:49,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:49,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:49,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:49,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:49,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:49,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:49,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:49,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:49,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:49,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:49,667 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,667 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-15 10:52:49,667 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.827458069757659E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-15 10:52:49,821 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 230 ms to try 5000 different iterations.  Found a solution that moves 18 regions; Going from a computed imbalance of 3.827458069757659E-5 to a new imbalance of 6.15811743360739E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.002); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,821 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv112585497:83 , srv1814917860:83 , srv572556615:83 , srv783597004:83 , srv101859221:84 , srv1038683419:84 , srv1040886835:84 , srv1097269675:84 , srv1105500464:84 , srv1182977905:84 , srv1186802307:84 , srv1207362753:84 , srv1219664427:84 , srv1237047127:84 , srv126019790:84 , srv1297468719:84 , srv1302693158:84 , srv1302792554:84 , srv1311775070:84 , srv1328250732:84 , srv1335056124:84 , srv1362740531:84 , srv1383434323:84 , srv141738160:84 , srv1422202552:84 , srv1460308037:84 , srv1471538066:84 , srv1489807596:84 , srv15139328:84 , srv1581696043:84 , srv1661761053:84 , srv1666568567:84 , srv1673489380:84 , srv1760265860:84 , srv1761688590:84 , srv1777569441:84 , srv1791437339:84 , srv1800350395:84 , srv1813390388:84 , srv1876115216:84 , srv1884148106:84 , srv1891241708:84 , srv1913776312:84 , srv1991791802:84 , srv1999301138:84 , srv2036817085:84 , srv2073815508:84 , srv2075708087:84 , srv2091648978:84 , srv2098530118:84 , srv2127754805:84 , srv2131750938:84 , srv233353144:84 , srv240410061:84 , srv240765783:84 , srv257219290:84 , srv287553087:84 , srv290959015:84 , srv31377852:84 , srv332178132:84 , srv362632965:84 , srv403982525:84 , srv409831432:84 , srv411185136:84 , srv428606718:84 , srv45229090:84 , srv47004007:84 , srv503838219:84 , srv521585099:84 , srv521973380:84 , srv547846007:84 , srv586738416:84 , srv590706231:84 , srv596568473:84 , srv60723814:84 , srv649443716:84 , srv700082097:84 , srv700967242:84 , srv711115555:84 , srv737371694:84 , srv750932553:84 , srv76681718:84 , srv782932398:84 , srv784796567:84 , srv851385851:84 , srv875472968:84 , srv877167561:84 , srv877735034:84 , srv8854124:84 , srv888372852:84 , srv903983232:84 , srv935624544:84 , srv943496727:84 , srv950482829:84 , srv967091409:84 , srv1640887290:85 , srv1727862448:85 , srv480232474:85 , srv832472129:85 , srv1478247474:684 }
2024-04-15 10:52:49,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:49,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv240410061=54, srv1813390388=38, srv877735034=92, srv1460308037=22, srv141738160=20, srv2075708087=48, srv1727862448=32, srv76681718=84, srv783597004=86, srv1489807596=25, srv1471538066=23, srv832472129=88, srv1207362753=8, srv2091648978=49, srv1991791802=44, srv1311775070=15, srv1328250732=16, srv1999301138=45, srv1302693158=13, srv2127754805=51, srv877167561=91, srv737371694=82, srv782932398=85, srv1673489380=31, srv572556615=73, srv1760265860=33, srv1791437339=36, srv126019790=11, srv2073815508=47, srv15139328=26, srv1876115216=40, srv1237047127=10, srv943496727=97, srv950482829=98, srv1761688590=34, srv31377852=59, srv711115555=81, srv1219664427=9, srv1661761053=29, srv47004007=67, srv1478247474=24, srv8854124=93, srv521585099=70, srv1186802307=7, srv649443716=78, srv1335056124=17, srv851385851=89, srv1182977905=6, srv480232474=68, srv1891241708=42, srv101859221=0, srv1302792554=14, srv521973380=71, srv1640887290=28, srv60723814=77, srv750932553=83, srv596568473=76, srv875472968=90, srv233353144=53, srv1040886835=2, srv287553087=57, srv240765783=55, srv586738416=74, srv403982525=62, srv1422202552=21, srv700082097=79, srv1097269675=3, srv1777569441=35, srv411185136=64, srv700967242=80, srv2098530118=50, srv1884148106=41, srv1581696043=27, srv903983232=95, srv935624544=96, srv1362740531=18, srv1383434323=19, srv503838219=69, srv1800350395=37, srv332178132=60, srv967091409=99, srv1666568567=30, srv112585497=5, srv888372852=94, srv1038683419=1, srv1913776312=43, srv1105500464=4, srv1814917860=39, srv428606718=65, srv1297468719=12, srv2036817085=46, srv784796567=87, srv409831432=63, srv45229090=66, srv590706231=75, srv2131750938=52, srv257219290=56, srv290959015=58, srv547846007=72, srv362632965=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:49,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:49,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:49,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:49,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:49,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-15 10:52:49,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-15 10:52:49,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-15 10:52:49,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-15 10:52:49,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-15 10:52:49,918 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,918 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv112585497:83 , srv1814917860:83 , srv572556615:83 , srv783597004:83 , srv101859221:84 , srv1038683419:84 , srv1040886835:84 , srv1097269675:84 , srv1105500464:84 , srv1182977905:84 , srv1186802307:84 , srv1207362753:84 , srv1219664427:84 , srv1237047127:84 , srv126019790:84 , srv1297468719:84 , srv1302693158:84 , srv1302792554:84 , srv1311775070:84 , srv1328250732:84 , srv1335056124:84 , srv1362740531:84 , srv1383434323:84 , srv141738160:84 , srv1422202552:84 , srv1460308037:84 , srv1471538066:84 , srv1489807596:84 , srv15139328:84 , srv1581696043:84 , srv1661761053:84 , srv1666568567:84 , srv1673489380:84 , srv1760265860:84 , srv1761688590:84 , srv1777569441:84 , srv1791437339:84 , srv1800350395:84 , srv1813390388:84 , srv1876115216:84 , srv1884148106:84 , srv1891241708:84 , srv1913776312:84 , srv1991791802:84 , srv1999301138:84 , srv2036817085:84 , srv2073815508:84 , srv2075708087:84 , srv2091648978:84 , srv2098530118:84 , srv2127754805:84 , srv2131750938:84 , srv233353144:84 , srv240410061:84 , srv240765783:84 , srv257219290:84 , srv287553087:84 , srv290959015:84 , srv31377852:84 , srv332178132:84 , srv362632965:84 , srv403982525:84 , srv409831432:84 , srv411185136:84 , srv428606718:84 , srv45229090:84 , srv47004007:84 , srv503838219:84 , srv521585099:84 , srv521973380:84 , srv547846007:84 , srv586738416:84 , srv590706231:84 , srv596568473:84 , srv60723814:84 , srv649443716:84 , srv700082097:84 , srv700967242:84 , srv711115555:84 , srv737371694:84 , srv750932553:84 , srv76681718:84 , srv782932398:84 , srv784796567:84 , srv851385851:84 , srv875472968:84 , srv877167561:84 , srv877735034:84 , srv8854124:84 , srv888372852:84 , srv903983232:84 , srv935624544:84 , srv943496727:84 , srv950482829:84 , srv967091409:84 , srv1640887290:85 , srv1727862448:85 , srv480232474:85 , srv832472129:85 , srv1478247474:684 }
2024-04-15 10:52:49,939 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=311 (was 311), ProcessCount=180 (was 180), AvailableMemoryMB=2652 (was 3324)
2024-04-15 10:52:49,946 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=311, ProcessCount=180, AvailableMemoryMB=2652
2024-04-15 10:52:49,947 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-15 10:52:49,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-15 10:52:49,954 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1081644581:84 , srv1118024387:84 , srv1155664538:84 , srv121973339:84 , srv1260215152:84 , srv1323050757:84 , srv1518147409:84 , srv1609220825:84 , srv1658231147:84 , srv1826382450:84 , srv1846241307:84 , srv1863813475:84 , srv1984063589:84 , srv2091488156:84 , srv230504497:84 , srv234449848:84 , srv247997550:84 , srv278354972:84 , srv292919044:84 , srv297127928:84 , srv439569804:84 , srv525205353:84 , srv630001635:84 , srv662038835:84 , srv680295480:84 , srv718071436:84 , srv741055206:84 , srv746942877:84 , srv840556597:84 , srv1241669580:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-15 10:52:49,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:52:49,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1826382450=10, srv2091488156=14, srv525205353=22, srv439569804=21, srv840556597=29, srv1155664538=2, srv278354972=18, srv630001635=23, srv746942877=28, srv1260215152=5, srv680295480=25, srv1323050757=6, srv230504497=15, srv718071436=26, srv247997550=17, srv1984063589=13, srv741055206=27, srv292919044=19, srv297127928=20, srv1518147409=7, srv234449848=16, srv1658231147=9, srv1863813475=12, srv1081644581=0, srv1609220825=8, srv1846241307=11, srv662038835=24, srv121973339=3, srv1118024387=1, srv1241669580=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:52:49,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:52:49,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-15 10:52:49,977 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0959939221045132, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:52:49,977 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912495516978342, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0959939221045132, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-15 10:54:49,967 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 8996322 different iterations.  Found a solution that moves 1848 regions; Going from a computed imbalance of 0.9912495516978342 to a new imbalance of 4.313324190526491E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:54:49,973 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1081644581:90 , srv1118024387:90 , srv1155664538:90 , srv121973339:90 , srv1241669580:90 , srv1260215152:90 , srv1323050757:90 , srv1518147409:90 , srv1609220825:90 , srv1658231147:90 , srv1826382450:90 , srv1846241307:90 , srv1863813475:90 , srv1984063589:90 , srv2091488156:90 , srv230504497:90 , srv234449848:90 , srv247997550:90 , srv278354972:90 , srv292919044:90 , srv297127928:90 , srv439569804:90 , srv525205353:90 , srv630001635:90 , srv662038835:90 , srv680295480:90 , srv718071436:90 , srv741055206:90 , srv746942877:90 , srv840556597:90 }
2024-04-15 10:54:49,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-15 10:54:49,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1826382450=10, srv2091488156=14, srv525205353=22, srv439569804=21, srv840556597=29, srv1155664538=2, srv278354972=18, srv630001635=23, srv746942877=28, srv1260215152=5, srv680295480=25, srv1323050757=6, srv230504497=15, srv718071436=26, srv247997550=17, srv1984063589=13, srv741055206=27, srv292919044=19, srv297127928=20, srv1518147409=7, srv234449848=16, srv1658231147=9, srv1863813475=12, srv1081644581=0, srv1609220825=8, srv1846241307=11, srv662038835=24, srv121973339=3, srv1118024387=1, srv1241669580=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-15 10:54:49,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-15 10:54:49,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-15 10:54:49,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-15 10:54:49,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-15 10:54:49,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-15 10:54:49,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-15 10:54:49,990 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-15 10:54:49,990 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1081644581:90 , srv1118024387:90 , srv1155664538:90 , srv121973339:90 , srv1241669580:90 , srv1260215152:90 , srv1323050757:90 , srv1518147409:90 , srv1609220825:90 , srv1658231147:90 , srv1826382450:90 , srv1846241307:90 , srv1863813475:90 , srv1984063589:90 , srv2091488156:90 , srv230504497:90 , srv234449848:90 , srv247997550:90 , srv278354972:90 , srv292919044:90 , srv297127928:90 , srv439569804:90 , srv525205353:90 , srv630001635:90 , srv662038835:90 , srv680295480:90 , srv718071436:90 , srv741055206:90 , srv746942877:90 , srv840556597:90 }
2024-04-15 10:54:50,003 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=270 (was 311), ProcessCount=179 (was 180), AvailableMemoryMB=2038 (was 2652)