2024-04-04 17:53:18,559 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-04 17:53:18,898 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-04 17:53:19,015 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-04 17:53:19,088 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-04 17:53:19,099 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-04 17:53:19,116 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=169, AvailableMemoryMB=4685
2024-04-04 17:53:19,119 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-04 17:53:19,119 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-04 17:53:19,263 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004830423:84 , srv1007315058:84 , srv1026973227:84 , srv1056113267:84 , srv112842419:84 , srv1137300611:84 , srv1140389639:84 , srv1150899619:84 , srv1168529876:84 , srv1173535487:84 , srv1174221108:84 , srv1210880688:84 , srv1217185020:84 , srv122430979:84 , srv1251751069:84 , srv1256476598:84 , srv13089580:84 , srv1327312807:84 , srv1332489632:84 , srv1369264149:84 , srv1389224992:84 , srv1410159895:84 , srv1429515449:84 , srv144722885:84 , srv1475113797:84 , srv1481315079:84 , srv1514527393:84 , srv1520777497:84 , srv1539891262:84 , srv1546176227:84 , srv1552206162:84 , srv1560601805:84 , srv1587696326:84 , srv1623057128:84 , srv1634748561:84 , srv1638786484:84 , srv1648183605:84 , srv168191415:84 , srv1712181151:84 , srv1730796939:84 , srv1732417215:84 , srv1752809457:84 , srv1771196984:84 , srv1842076716:84 , srv1842487178:84 , srv1869449251:84 , srv188174832:84 , srv1929461979:84 , srv1969599917:84 , srv1975589207:84 , srv1978310274:84 , srv1996300920:84 , srv2003486382:84 , srv202454708:84 , srv2028217209:84 , srv2063102848:84 , srv2065904898:84 , srv2095969337:84 , srv2146483834:84 , srv234623435:84 , srv24737339:84 , srv257742810:84 , srv283430260:84 , srv304292350:84 , srv314487217:84 , srv34033265:84 , srv453211276:84 , srv454551196:84 , srv46270494:84 , srv466937991:84 , srv467813312:84 , srv469135735:84 , srv484935408:84 , srv519343215:84 , srv590740501:84 , srv626252404:84 , srv632876311:84 , srv639944408:84 , srv659961007:84 , srv679954366:84 , srv690626330:84 , srv693882978:84 , srv700930636:84 , srv706316493:84 , srv716133369:84 , srv75517387:84 , srv769683706:84 , srv827703520:84 , srv831160692:84 , srv834071965:84 , srv853302513:84 , srv856262730:84 , srv874826023:84 , srv877013081:84 , srv903887255:84 , srv907110343:84 , srv923707599:84 , srv929414107:84 , srv977318845:84 , srv1419791954:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-04 17:53:19,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:19,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1174221108=10, srv1978310274=51, srv1730796939=40, srv1520777497=28, srv1869449251=46, srv34033265=66, srv1842487178=45, srv1150899619=7, srv907110343=96, srv2028217209=55, srv2063102848=56, srv626252404=76, srv1546176227=30, srv1752809457=42, srv304292350=64, srv632876311=77, srv314487217=65, srv834071965=90, srv122430979=13, srv903887255=95, srv1732417215=41, srv1007315058=1, srv1929461979=48, srv1168529876=8, srv168191415=38, srv453211276=67, srv112842419=4, srv1140389639=6, srv831160692=89, srv923707599=97, srv1327312807=17, srv827703520=88, srv454551196=68, srv46270494=69, srv1056113267=3, srv1552206162=31, srv639944408=78, srv1210880688=11, srv144722885=24, srv853302513=91, srv234623435=60, srv1256476598=15, srv1634748561=35, srv1996300920=52, srv977318845=99, srv769683706=87, srv2146483834=59, srv693882978=82, srv283430260=63, srv484935408=73, srv1173535487=9, srv75517387=86, srv1475113797=25, srv706316493=84, srv1217185020=12, srv1514527393=27, srv2065904898=57, srv1975589207=50, srv877013081=94, srv1560601805=32, srv700930636=83, srv2095969337=58, srv1137300611=5, srv856262730=92, srv202454708=54, srv1623057128=34, srv1587696326=33, srv24737339=61, srv716133369=85, srv1004830423=0, srv467813312=71, srv1251751069=14, srv1771196984=43, srv1026973227=2, srv1842076716=44, srv1429515449=23, srv690626330=81, srv874826023=93, srv2003486382=53, srv1638786484=36, srv13089580=16, srv257742810=62, srv679954366=80, srv1481315079=26, srv1969599917=49, srv466937991=70, srv188174832=47, srv1369264149=19, srv1539891262=29, srv1712181151=39, srv590740501=75, srv659961007=79, srv1419791954=22, srv1410159895=21, srv519343215=74, srv1332489632=18, srv929414107=98, srv1648183605=37, srv469135735=72, srv1389224992=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:19,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:19,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:19,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:19,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:19,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:19,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-04 17:53:19,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-04 17:53:19,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-04 17:53:19,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-04 17:53:19,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-04 17:53:19,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-04 17:53:19,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-04 17:53:19,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-04 17:53:19,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-04 17:53:19,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-04 17:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-04 17:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-04 17:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-04 17:53:19,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-04 17:53:19,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-04 17:53:19,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-04 17:53:19,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-04 17:53:19,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-04 17:53:19,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-04 17:53:19,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-04 17:53:19,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-04 17:53:19,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-04 17:53:19,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-04 17:53:19,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-04 17:53:19,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:19,842 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-04 17:53:19,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-04 17:53:20,329 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 903 ms to try 5000 different iterations.  Found a solution that moves 4701 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1762358654915745. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04030189107526375, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05440010563137562, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5223333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04030189107526375, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:20,343 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1150899619:82 , srv2063102848:82 , srv769683706:82 , srv907110343:82 , srv1056113267:83 , srv1168529876:83 , srv1256476598:83 , srv1327312807:83 , srv144722885:83 , srv1771196984:83 , srv469135735:83 , srv484935408:83 , srv75517387:83 , srv1026973227:84 , srv112842419:84 , srv1712181151:84 , srv1978310274:84 , srv2003486382:84 , srv2065904898:84 , srv831160692:84 , srv874826023:84 , srv1007315058:85 , srv13089580:85 , srv1332489632:85 , srv1389224992:85 , srv1410159895:85 , srv1429515449:85 , srv1475113797:85 , srv1514527393:85 , srv1546176227:85 , srv1560601805:85 , srv1587696326:85 , srv168191415:85 , srv1842487178:85 , srv467813312:85 , srv590740501:85 , srv626252404:85 , srv639944408:85 , srv659961007:85 , srv706316493:85 , srv877013081:85 , srv977318845:85 , srv1140389639:86 , srv1217185020:86 , srv122430979:86 , srv1520777497:86 , srv1539891262:86 , srv1552206162:86 , srv1623057128:86 , srv1638786484:86 , srv1842076716:86 , srv1869449251:86 , srv202454708:86 , srv24737339:86 , srv304292350:86 , srv34033265:86 , srv46270494:86 , srv632876311:86 , srv700930636:86 , srv716133369:86 , srv856262730:86 , srv1004830423:87 , srv1137300611:87 , srv1210880688:87 , srv1251751069:87 , srv1369264149:87 , srv1648183605:87 , srv1732417215:87 , srv1996300920:87 , srv2028217209:87 , srv2095969337:87 , srv2146483834:87 , srv283430260:87 , srv679954366:87 , srv853302513:87 , srv1730796939:88 , srv314487217:88 , srv454551196:88 , srv827703520:88 , srv903887255:88 , srv929414107:88 , srv1173535487:89 , srv1634748561:89 , srv188174832:89 , srv234623435:89 , srv690626330:89 , srv693882978:89 , srv834071965:89 , srv453211276:90 , srv1174221108:91 , srv1969599917:91 , srv1975589207:91 , srv1481315079:92 , srv1929461979:92 , srv923707599:92 , srv1752809457:93 , srv519343215:93 , srv466937991:94 , srv257742810:98 , srv1419791954:448 }
2024-04-04 17:53:20,349 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:20,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1174221108=10, srv1978310274=51, srv1730796939=40, srv1520777497=28, srv1869449251=46, srv34033265=66, srv1842487178=45, srv1150899619=7, srv907110343=96, srv2028217209=55, srv2063102848=56, srv626252404=76, srv1546176227=30, srv1752809457=42, srv304292350=64, srv632876311=77, srv314487217=65, srv834071965=90, srv122430979=13, srv903887255=95, srv1732417215=41, srv1007315058=1, srv1929461979=48, srv1168529876=8, srv168191415=38, srv453211276=67, srv112842419=4, srv1140389639=6, srv831160692=89, srv923707599=97, srv1327312807=17, srv827703520=88, srv454551196=68, srv46270494=69, srv1056113267=3, srv1552206162=31, srv639944408=78, srv1210880688=11, srv144722885=24, srv853302513=91, srv234623435=60, srv1256476598=15, srv1634748561=35, srv1996300920=52, srv977318845=99, srv769683706=87, srv2146483834=59, srv693882978=82, srv283430260=63, srv484935408=73, srv1173535487=9, srv75517387=86, srv1475113797=25, srv706316493=84, srv1217185020=12, srv1514527393=27, srv2065904898=57, srv1975589207=50, srv877013081=94, srv1560601805=32, srv700930636=83, srv2095969337=58, srv1137300611=5, srv856262730=92, srv202454708=54, srv1623057128=34, srv1587696326=33, srv24737339=61, srv716133369=85, srv1004830423=0, srv467813312=71, srv1251751069=14, srv1771196984=43, srv1026973227=2, srv1842076716=44, srv1429515449=23, srv690626330=81, srv874826023=93, srv2003486382=53, srv1638786484=36, srv13089580=16, srv257742810=62, srv679954366=80, srv1481315079=26, srv1969599917=49, srv466937991=70, srv188174832=47, srv1369264149=19, srv1539891262=29, srv1712181151=39, srv590740501=75, srv659961007=79, srv1419791954=22, srv1410159895=21, srv519343215=74, srv1332489632=18, srv929414107=98, srv1648183605=37, srv469135735=72, srv1389224992=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:20,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:20,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:20,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:20,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:20,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:20,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-04 17:53:20,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-04 17:53:20,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-04 17:53:20,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-04 17:53:20,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-04 17:53:20,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-04 17:53:20,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-04 17:53:20,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-04 17:53:20,554 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04030189107526375, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05440010563137562, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04030189107526375, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:20,554 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-04 17:53:20,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1762029483860231, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04030189107526375, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05440010563137562, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04030189107526375, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23816666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-04 17:53:20,899 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 2643 regions; Going from a computed imbalance of 0.1762029483860231 to a new imbalance of 0.014188991079620194. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021627277211682738); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03155527101877691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2936666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021627277211682738); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:20,912 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv257742810:74 , srv466937991:79 , srv923707599:79 , srv1481315079:80 , srv1929461979:80 , srv453211276:80 , srv1969599917:81 , srv519343215:81 , srv1174221108:82 , srv1975589207:82 , srv314487217:82 , srv827703520:82 , srv1173535487:83 , srv1634748561:83 , srv1752809457:83 , srv283430260:83 , srv1004830423:84 , srv1389224992:84 , srv1648183605:84 , srv1730796939:84 , srv1732417215:84 , srv234623435:84 , srv304292350:84 , srv679954366:84 , srv690626330:84 , srv700930636:84 , srv903887255:84 , srv1026973227:85 , srv112842419:85 , srv1137300611:85 , srv1140389639:85 , srv1210880688:85 , srv122430979:85 , srv1560601805:85 , srv1869449251:85 , srv1996300920:85 , srv2028217209:85 , srv626252404:85 , srv834071965:85 , srv1217185020:86 , srv13089580:86 , srv1327312807:86 , srv1369264149:86 , srv1429515449:86 , srv144722885:86 , srv1514527393:86 , srv1552206162:86 , srv1587696326:86 , srv1638786484:86 , srv188174832:86 , srv2095969337:86 , srv24737339:86 , srv34033265:86 , srv454551196:86 , srv467813312:86 , srv639944408:86 , srv693882978:86 , srv706316493:86 , srv716133369:86 , srv853302513:86 , srv856262730:86 , srv874826023:86 , srv929414107:86 , srv1007315058:87 , srv1056113267:87 , srv1251751069:87 , srv1332489632:87 , srv1623057128:87 , srv1842076716:87 , srv1978310274:87 , srv202454708:87 , srv590740501:87 , srv632876311:87 , srv75517387:87 , srv831160692:87 , srv877013081:87 , srv1150899619:88 , srv1168529876:88 , srv1475113797:88 , srv168191415:88 , srv2063102848:88 , srv2065904898:88 , srv2146483834:88 , srv46270494:88 , srv1539891262:89 , srv1546176227:89 , srv1712181151:89 , srv1842487178:89 , srv2003486382:89 , srv484935408:89 , srv769683706:89 , srv1256476598:90 , srv1520777497:90 , srv1771196984:90 , srv659961007:90 , srv907110343:90 , srv977318845:90 , srv1410159895:91 , srv469135735:91 , srv1419791954:518 }
2024-04-04 17:53:20,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:20,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1174221108=10, srv1978310274=51, srv1730796939=40, srv1520777497=28, srv1869449251=46, srv34033265=66, srv1842487178=45, srv1150899619=7, srv907110343=96, srv2028217209=55, srv2063102848=56, srv626252404=76, srv1546176227=30, srv1752809457=42, srv304292350=64, srv632876311=77, srv314487217=65, srv834071965=90, srv122430979=13, srv903887255=95, srv1732417215=41, srv1007315058=1, srv1929461979=48, srv1168529876=8, srv168191415=38, srv453211276=67, srv112842419=4, srv1140389639=6, srv831160692=89, srv923707599=97, srv1327312807=17, srv827703520=88, srv454551196=68, srv46270494=69, srv1056113267=3, srv1552206162=31, srv639944408=78, srv1210880688=11, srv144722885=24, srv853302513=91, srv234623435=60, srv1256476598=15, srv1634748561=35, srv1996300920=52, srv977318845=99, srv769683706=87, srv2146483834=59, srv693882978=82, srv283430260=63, srv484935408=73, srv1173535487=9, srv75517387=86, srv1475113797=25, srv706316493=84, srv1217185020=12, srv1514527393=27, srv2065904898=57, srv1975589207=50, srv877013081=94, srv1560601805=32, srv700930636=83, srv2095969337=58, srv1137300611=5, srv856262730=92, srv202454708=54, srv1623057128=34, srv1587696326=33, srv24737339=61, srv716133369=85, srv1004830423=0, srv467813312=71, srv1251751069=14, srv1771196984=43, srv1026973227=2, srv1842076716=44, srv1429515449=23, srv690626330=81, srv874826023=93, srv2003486382=53, srv1638786484=36, srv13089580=16, srv257742810=62, srv679954366=80, srv1481315079=26, srv1969599917=49, srv466937991=70, srv188174832=47, srv1369264149=19, srv1539891262=29, srv1712181151=39, srv590740501=75, srv659961007=79, srv1419791954=22, srv1410159895=21, srv519343215=74, srv1332489632=18, srv929414107=98, srv1648183605=37, srv469135735=72, srv1389224992=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:20,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:20,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:20,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-04 17:53:20,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-04 17:53:20,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-04 17:53:20,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-04 17:53:20,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-04 17:53:20,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-04 17:53:20,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-04 17:53:20,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-04 17:53:20,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-04 17:53:20,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-04 17:53:20,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-04 17:53:20,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-04 17:53:21,083 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021627277211682738); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03155527101877691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021627277211682738); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:21,083 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-04 17:53:21,083 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014170484398068959, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021627277211682738); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03155527101877691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021627277211682738); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-04 17:53:21,382 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 408 ms to try 5000 different iterations.  Found a solution that moves 985 regions; Going from a computed imbalance of 0.014170484398068959 to a new imbalance of 5.826771601836709E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005322847904476634); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.10944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005322847904476634); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:21,383 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1752809457:81 , srv1410159895:82 , srv1520777497:82 , srv659961007:82 , srv977318845:82 , srv1539891262:83 , srv1546176227:83 , srv188174832:83 , srv2146483834:83 , srv454551196:83 , srv46270494:83 , srv693882978:83 , srv834071965:83 , srv1251751069:84 , srv1256476598:84 , srv1623057128:84 , srv1712181151:84 , srv1842076716:84 , srv1842487178:84 , srv1975589207:84 , srv2003486382:84 , srv2095969337:84 , srv466937991:84 , srv469135735:84 , srv519343215:84 , srv632876311:84 , srv690626330:84 , srv929414107:84 , srv1137300611:85 , srv1173535487:85 , srv1210880688:85 , srv1217185020:85 , srv1332489632:85 , srv1369264149:85 , srv1771196984:85 , srv202454708:85 , srv2065904898:85 , srv234623435:85 , srv24737339:85 , srv257742810:85 , srv34033265:85 , srv590740501:85 , srv716133369:85 , srv856262730:85 , srv907110343:85 , srv1004830423:86 , srv1007315058:86 , srv1140389639:86 , srv1168529876:86 , srv1174221108:86 , srv122430979:86 , srv13089580:86 , srv1475113797:86 , srv1481315079:86 , srv1552206162:86 , srv1587696326:86 , srv1634748561:86 , srv1638786484:86 , srv1648183605:86 , srv168191415:86 , srv1869449251:86 , srv1929461979:86 , srv1978310274:86 , srv1996300920:86 , srv2028217209:86 , srv484935408:86 , srv831160692:86 , srv853302513:86 , srv903887255:86 , srv923707599:86 , srv1056113267:87 , srv1150899619:87 , srv1429515449:87 , srv1514527393:87 , srv1560601805:87 , srv1730796939:87 , srv1732417215:87 , srv1969599917:87 , srv2063102848:87 , srv283430260:87 , srv453211276:87 , srv467813312:87 , srv626252404:87 , srv639944408:87 , srv679954366:87 , srv75517387:87 , srv769683706:87 , srv874826023:87 , srv877013081:87 , srv1026973227:88 , srv112842419:88 , srv1327312807:88 , srv1389224992:88 , srv144722885:88 , srv304292350:88 , srv314487217:88 , srv700930636:88 , srv706316493:88 , srv827703520:88 , srv1419791954:539 }
2024-04-04 17:53:21,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:21,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1174221108=10, srv1978310274=51, srv1730796939=40, srv1520777497=28, srv1869449251=46, srv34033265=66, srv1842487178=45, srv1150899619=7, srv907110343=96, srv2028217209=55, srv2063102848=56, srv626252404=76, srv1546176227=30, srv1752809457=42, srv304292350=64, srv632876311=77, srv314487217=65, srv834071965=90, srv122430979=13, srv903887255=95, srv1732417215=41, srv1007315058=1, srv1929461979=48, srv1168529876=8, srv168191415=38, srv453211276=67, srv112842419=4, srv1140389639=6, srv831160692=89, srv923707599=97, srv1327312807=17, srv827703520=88, srv454551196=68, srv46270494=69, srv1056113267=3, srv1552206162=31, srv639944408=78, srv1210880688=11, srv144722885=24, srv853302513=91, srv234623435=60, srv1256476598=15, srv1634748561=35, srv1996300920=52, srv977318845=99, srv769683706=87, srv2146483834=59, srv693882978=82, srv283430260=63, srv484935408=73, srv1173535487=9, srv75517387=86, srv1475113797=25, srv706316493=84, srv1217185020=12, srv1514527393=27, srv2065904898=57, srv1975589207=50, srv877013081=94, srv1560601805=32, srv700930636=83, srv2095969337=58, srv1137300611=5, srv856262730=92, srv202454708=54, srv1623057128=34, srv1587696326=33, srv24737339=61, srv716133369=85, srv1004830423=0, srv467813312=71, srv1251751069=14, srv1771196984=43, srv1026973227=2, srv1842076716=44, srv1429515449=23, srv690626330=81, srv874826023=93, srv2003486382=53, srv1638786484=36, srv13089580=16, srv257742810=62, srv679954366=80, srv1481315079=26, srv1969599917=49, srv466937991=70, srv188174832=47, srv1369264149=19, srv1539891262=29, srv1712181151=39, srv590740501=75, srv659961007=79, srv1419791954=22, srv1410159895=21, srv519343215=74, srv1332489632=18, srv929414107=98, srv1648183605=37, srv469135735=72, srv1389224992=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:21,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:21,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:21,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-04 17:53:21,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-04 17:53:21,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-04 17:53:21,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-04 17:53:21,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-04 17:53:21,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-04 17:53:21,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-04 17:53:21,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-04 17:53:21,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-04 17:53:21,571 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005322847904476634); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005322847904476634); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:21,572 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-04 17:53:21,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.757800427686245E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005322847904476634); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00721639236343101); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005322847904476634); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-04 17:53:21,982 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 554 ms to try 5000 different iterations.  Found a solution that moves 230 regions; Going from a computed imbalance of 5.757800427686245E-4 to a new imbalance of 1.7886900707870763E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.025555555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:21,983 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1174221108:83 , srv1475113797:83 , srv1481315079:83 , srv168191415:83 , srv1730796939:83 , srv1969599917:83 , srv706316493:83 , srv877013081:83 , srv1007315058:84 , srv1251751069:84 , srv1369264149:84 , srv1410159895:84 , srv1429515449:84 , srv1514527393:84 , srv1520777497:84 , srv1539891262:84 , srv1552206162:84 , srv1634748561:84 , srv1638786484:84 , srv1732417215:84 , srv1771196984:84 , srv1842487178:84 , srv188174832:84 , srv1929461979:84 , srv1978310274:84 , srv1996300920:84 , srv202454708:84 , srv2028217209:84 , srv2095969337:84 , srv2146483834:84 , srv234623435:84 , srv304292350:84 , srv314487217:84 , srv467813312:84 , srv469135735:84 , srv484935408:84 , srv519343215:84 , srv639944408:84 , srv679954366:84 , srv693882978:84 , srv700930636:84 , srv75517387:84 , srv769683706:84 , srv827703520:84 , srv853302513:84 , srv903887255:84 , srv929414107:84 , srv977318845:84 , srv1004830423:85 , srv1026973227:85 , srv1056113267:85 , srv112842419:85 , srv1137300611:85 , srv1140389639:85 , srv1150899619:85 , srv1168529876:85 , srv1173535487:85 , srv1210880688:85 , srv1217185020:85 , srv122430979:85 , srv1256476598:85 , srv13089580:85 , srv1327312807:85 , srv1332489632:85 , srv1389224992:85 , srv144722885:85 , srv1546176227:85 , srv1560601805:85 , srv1587696326:85 , srv1623057128:85 , srv1648183605:85 , srv1712181151:85 , srv1752809457:85 , srv1842076716:85 , srv1869449251:85 , srv2003486382:85 , srv2063102848:85 , srv24737339:85 , srv257742810:85 , srv283430260:85 , srv34033265:85 , srv453211276:85 , srv454551196:85 , srv46270494:85 , srv466937991:85 , srv590740501:85 , srv626252404:85 , srv632876311:85 , srv659961007:85 , srv716133369:85 , srv834071965:85 , srv856262730:85 , srv874826023:85 , srv907110343:85 , srv923707599:85 , srv1975589207:86 , srv2065904898:86 , srv690626330:86 , srv831160692:86 , srv1419791954:637 }
2024-04-04 17:53:21,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:21,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1174221108=10, srv1978310274=51, srv1730796939=40, srv1520777497=28, srv1869449251=46, srv34033265=66, srv1842487178=45, srv1150899619=7, srv907110343=96, srv2028217209=55, srv2063102848=56, srv626252404=76, srv1546176227=30, srv1752809457=42, srv304292350=64, srv632876311=77, srv314487217=65, srv834071965=90, srv122430979=13, srv903887255=95, srv1732417215=41, srv1007315058=1, srv1929461979=48, srv1168529876=8, srv168191415=38, srv453211276=67, srv112842419=4, srv1140389639=6, srv831160692=89, srv923707599=97, srv1327312807=17, srv827703520=88, srv454551196=68, srv46270494=69, srv1056113267=3, srv1552206162=31, srv639944408=78, srv1210880688=11, srv144722885=24, srv853302513=91, srv234623435=60, srv1256476598=15, srv1634748561=35, srv1996300920=52, srv977318845=99, srv769683706=87, srv2146483834=59, srv693882978=82, srv283430260=63, srv484935408=73, srv1173535487=9, srv75517387=86, srv1475113797=25, srv706316493=84, srv1217185020=12, srv1514527393=27, srv2065904898=57, srv1975589207=50, srv877013081=94, srv1560601805=32, srv700930636=83, srv2095969337=58, srv1137300611=5, srv856262730=92, srv202454708=54, srv1623057128=34, srv1587696326=33, srv24737339=61, srv716133369=85, srv1004830423=0, srv467813312=71, srv1251751069=14, srv1771196984=43, srv1026973227=2, srv1842076716=44, srv1429515449=23, srv690626330=81, srv874826023=93, srv2003486382=53, srv1638786484=36, srv13089580=16, srv257742810=62, srv679954366=80, srv1481315079=26, srv1969599917=49, srv466937991=70, srv188174832=47, srv1369264149=19, srv1539891262=29, srv1712181151=39, srv590740501=75, srv659961007=79, srv1419791954=22, srv1410159895=21, srv519343215=74, srv1332489632=18, srv929414107=98, srv1648183605=37, srv469135735=72, srv1389224992=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:22,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:22,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:22,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:22,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:22,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:22,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:22,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-04 17:53:22,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-04 17:53:22,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-04 17:53:22,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-04 17:53:22,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-04 17:53:22,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-04 17:53:22,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-04 17:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-04 17:53:22,121 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.6276406286083273E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:22,121 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1174221108:83 , srv1475113797:83 , srv1481315079:83 , srv168191415:83 , srv1730796939:83 , srv1969599917:83 , srv706316493:83 , srv877013081:83 , srv1007315058:84 , srv1251751069:84 , srv1369264149:84 , srv1410159895:84 , srv1429515449:84 , srv1514527393:84 , srv1520777497:84 , srv1539891262:84 , srv1552206162:84 , srv1634748561:84 , srv1638786484:84 , srv1732417215:84 , srv1771196984:84 , srv1842487178:84 , srv188174832:84 , srv1929461979:84 , srv1978310274:84 , srv1996300920:84 , srv202454708:84 , srv2028217209:84 , srv2095969337:84 , srv2146483834:84 , srv234623435:84 , srv304292350:84 , srv314487217:84 , srv467813312:84 , srv469135735:84 , srv484935408:84 , srv519343215:84 , srv639944408:84 , srv679954366:84 , srv693882978:84 , srv700930636:84 , srv75517387:84 , srv769683706:84 , srv827703520:84 , srv853302513:84 , srv903887255:84 , srv929414107:84 , srv977318845:84 , srv1004830423:85 , srv1026973227:85 , srv1056113267:85 , srv112842419:85 , srv1137300611:85 , srv1140389639:85 , srv1150899619:85 , srv1168529876:85 , srv1173535487:85 , srv1210880688:85 , srv1217185020:85 , srv122430979:85 , srv1256476598:85 , srv13089580:85 , srv1327312807:85 , srv1332489632:85 , srv1389224992:85 , srv144722885:85 , srv1546176227:85 , srv1560601805:85 , srv1587696326:85 , srv1623057128:85 , srv1648183605:85 , srv1712181151:85 , srv1752809457:85 , srv1842076716:85 , srv1869449251:85 , srv2003486382:85 , srv2063102848:85 , srv24737339:85 , srv257742810:85 , srv283430260:85 , srv34033265:85 , srv453211276:85 , srv454551196:85 , srv46270494:85 , srv466937991:85 , srv590740501:85 , srv626252404:85 , srv632876311:85 , srv659961007:85 , srv716133369:85 , srv834071965:85 , srv856262730:85 , srv874826023:85 , srv907110343:85 , srv923707599:85 , srv1975589207:86 , srv2065904898:86 , srv690626330:86 , srv831160692:86 , srv1419791954:637 }
2024-04-04 17:53:22,156 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=277 (was 277), ProcessCount=169 (was 169), AvailableMemoryMB=3953 (was 4685)
2024-04-04 17:53:22,166 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=169, AvailableMemoryMB=3953
2024-04-04 17:53:22,167 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-04 17:53:22,167 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-04 17:53:22,181 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1054549794:84 , srv116243378:84 , srv1360850244:84 , srv1440765732:84 , srv1471543669:84 , srv1716791084:84 , srv1793727548:84 , srv1794032522:84 , srv1846735179:84 , srv1878939927:84 , srv195777617:84 , srv2090207899:84 , srv286421583:84 , srv306417001:84 , srv332672610:84 , srv388369400:84 , srv473932531:84 , srv507239275:84 , srv53487605:84 , srv575866383:84 , srv607411824:84 , srv647902888:84 , srv654291594:84 , srv758432072:84 , srv864704046:84 , srv896422618:84 , srv92483264:84 , srv986882722:84 , srv998194989:84 , srv322851544:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-04 17:53:22,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:53:22,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv388369400=16, srv1846735179=8, srv507239275=18, srv116243378=1, srv306417001=13, srv758432072=24, srv92483264=27, srv1471543669=4, srv322851544=14, srv654291594=23, srv896422618=26, srv1360850244=2, srv1716791084=5, srv1054549794=0, srv1440765732=3, srv647902888=22, srv53487605=19, srv1794032522=7, srv575866383=20, srv864704046=25, srv2090207899=11, srv332672610=15, srv1878939927=9, srv1793727548=6, srv607411824=21, srv986882722=28, srv286421583=12, srv473932531=17, srv195777617=10, srv998194989=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:53:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:53:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:53:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:53:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:53:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:53:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:53:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:53:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:53:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-04 17:53:22,219 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1263613917430142, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:53:22,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912591203883583, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1263613917430142, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-04 17:55:22,204 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5517264 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912591203883583 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:55:22,211 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1054549794:90 , srv116243378:90 , srv1360850244:90 , srv1440765732:90 , srv1471543669:90 , srv1716791084:90 , srv1793727548:90 , srv1794032522:90 , srv1846735179:90 , srv1878939927:90 , srv195777617:90 , srv2090207899:90 , srv286421583:90 , srv306417001:90 , srv322851544:90 , srv332672610:90 , srv388369400:90 , srv473932531:90 , srv507239275:90 , srv53487605:90 , srv575866383:90 , srv607411824:90 , srv647902888:90 , srv654291594:90 , srv758432072:90 , srv864704046:90 , srv896422618:90 , srv92483264:90 , srv986882722:90 , srv998194989:90 }
2024-04-04 17:55:22,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-04 17:55:22,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv388369400=16, srv1846735179=8, srv507239275=18, srv116243378=1, srv306417001=13, srv758432072=24, srv92483264=27, srv1471543669=4, srv322851544=14, srv654291594=23, srv896422618=26, srv1360850244=2, srv1716791084=5, srv1054549794=0, srv1440765732=3, srv647902888=22, srv53487605=19, srv1794032522=7, srv575866383=20, srv864704046=25, srv2090207899=11, srv332672610=15, srv1878939927=9, srv1793727548=6, srv607411824=21, srv986882722=28, srv286421583=12, srv473932531=17, srv195777617=10, srv998194989=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-04 17:55:22,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-04 17:55:22,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-04 17:55:22,232 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-04 17:55:22,232 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1054549794:90 , srv116243378:90 , srv1360850244:90 , srv1440765732:90 , srv1471543669:90 , srv1716791084:90 , srv1793727548:90 , srv1794032522:90 , srv1846735179:90 , srv1878939927:90 , srv195777617:90 , srv2090207899:90 , srv286421583:90 , srv306417001:90 , srv322851544:90 , srv332672610:90 , srv388369400:90 , srv473932531:90 , srv507239275:90 , srv53487605:90 , srv575866383:90 , srv607411824:90 , srv647902888:90 , srv654291594:90 , srv758432072:90 , srv864704046:90 , srv896422618:90 , srv92483264:90 , srv986882722:90 , srv998194989:90 }
2024-04-04 17:55:22,246 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 277), ProcessCount=167 (was 169), AvailableMemoryMB=3159 (was 3953)