2023-10-19 21:56:59,590 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-19 21:56:59,892 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-19 21:57:00,015 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-19 21:57:00,087 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 21:57:00,095 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-19 21:57:00,110 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=220, ProcessCount=168, AvailableMemoryMB=7257
2023-10-19 21:57:00,112 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 21:57:00,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-19 21:57:00,260 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv106610178:84 , srv1121732440:84 , srv1133352723:84 , srv1153084798:84 , srv1161805877:84 , srv1195740816:84 , srv1214491791:84 , srv1214970818:84 , srv1264809760:84 , srv1294126017:84 , srv132287426:84 , srv1323085712:84 , srv1363247492:84 , srv1379545824:84 , srv1382150480:84 , srv1420395625:84 , srv1425457541:84 , srv1438414460:84 , srv1450479750:84 , srv1466955296:84 , srv1542996779:84 , srv1543047617:84 , srv1546443560:84 , srv1555715886:84 , srv157205079:84 , srv1586439184:84 , srv1611931384:84 , srv1617502505:84 , srv1624749773:84 , srv1642480319:84 , srv1657681266:84 , srv1675875015:84 , srv16854906:84 , srv1693327223:84 , srv1718003531:84 , srv1718829405:84 , srv1782688234:84 , srv1797120305:84 , srv1845922243:84 , srv1903268418:84 , srv1906260288:84 , srv1926098467:84 , srv1940770336:84 , srv1948551866:84 , srv1955425326:84 , srv1988610093:84 , srv2037068909:84 , srv206213443:84 , srv20694211:84 , srv2070528577:84 , srv2074465158:84 , srv214542821:84 , srv252486933:84 , srv268694417:84 , srv283110205:84 , srv301039938:84 , srv303831264:84 , srv305378693:84 , srv317197783:84 , srv324173231:84 , srv334102726:84 , srv334710303:84 , srv33601688:84 , srv348580709:84 , srv360434320:84 , srv383322525:84 , srv407287955:84 , srv412632992:84 , srv446042689:84 , srv474497597:84 , srv483326812:84 , srv516643137:84 , srv527307780:84 , srv529885738:84 , srv55801803:84 , srv573167608:84 , srv599905775:84 , srv620199996:84 , srv627045543:84 , srv635977096:84 , srv639129758:84 , srv639170583:84 , srv643326428:84 , srv666866736:84 , srv684856381:84 , srv719016074:84 , srv770472355:84 , srv770640459:84 , srv775174070:84 , srv781471636:84 , srv831870762:84 , srv83643112:84 , srv850812151:84 , srv871194874:84 , srv888528300:84 , srv896138853:84 , srv909234618:84 , srv928972931:84 , srv931502993:84 , srv125490865:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-19 21:57:00,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:00,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:00,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:00,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:00,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:00,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:00,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:00,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:00,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:00,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:00,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:00,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:00,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:00,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:00,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:00,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:00,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:00,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:00,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:00,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:00,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:00,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:00,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:00,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:00,560 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:00,560 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 21:57:00,560 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 21:57:00,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 450 ms to try 5000 different iterations.  Found a solution that moves 4608 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18857771121555067. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04216340637344303, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05767667922652384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.512, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04216340637344303, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:00,879 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv252486933:82 , srv635977096:82 , srv1121732440:83 , srv1214970818:83 , srv1624749773:83 , srv1642480319:83 , srv1782688234:83 , srv1940770336:83 , srv317197783:83 , srv334710303:83 , srv407287955:83 , srv871194874:83 , srv1214491791:84 , srv1379545824:84 , srv1611931384:84 , srv1657681266:84 , srv1693327223:84 , srv1903268418:84 , srv214542821:84 , srv348580709:84 , srv627045543:84 , srv684856381:84 , srv1153084798:85 , srv1264809760:85 , srv1363247492:85 , srv1543047617:85 , srv1555715886:85 , srv157205079:85 , srv1586439184:85 , srv1845922243:85 , srv1988610093:85 , srv206213443:85 , srv283110205:85 , srv303831264:85 , srv324173231:85 , srv334102726:85 , srv383322525:85 , srv474497597:85 , srv529885738:85 , srv639170583:85 , srv928972931:85 , srv931502993:85 , srv1323085712:86 , srv1382150480:86 , srv1438414460:86 , srv1450479750:86 , srv1542996779:86 , srv1718829405:86 , srv1797120305:86 , srv305378693:86 , srv527307780:86 , srv639129758:86 , srv770640459:86 , srv83643112:86 , srv106610178:87 , srv1133352723:87 , srv1195740816:87 , srv1294126017:87 , srv1420395625:87 , srv1466955296:87 , srv1617502505:87 , srv1718003531:87 , srv1926098467:87 , srv2070528577:87 , srv33601688:87 , srv446042689:87 , srv483326812:87 , srv620199996:87 , srv643326428:87 , srv666866736:87 , srv719016074:87 , srv781471636:87 , srv831870762:87 , srv850812151:87 , srv888528300:87 , srv896138853:87 , srv909234618:87 , srv132287426:88 , srv1948551866:88 , srv20694211:88 , srv412632992:88 , srv516643137:88 , srv770472355:88 , srv775174070:88 , srv1161805877:89 , srv1425457541:89 , srv16854906:89 , srv2037068909:89 , srv2074465158:89 , srv301039938:89 , srv599905775:89 , srv1546443560:90 , srv1906260288:90 , srv1955425326:90 , srv1675875015:91 , srv360434320:91 , srv55801803:91 , srv268694417:93 , srv573167608:93 , srv125490865:465 }
2023-10-19 21:57:00,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:00,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:00,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:00,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:00,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:00,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:00,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:00,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:00,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:00,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:00,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:00,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:01,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04216340637344303, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05767667922652384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04216340637344303, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,035 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 21:57:01,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18854544530991765, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04216340637344303, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05767667922652384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04216340637344303, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 21:57:01,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 279 ms to try 5000 different iterations.  Found a solution that moves 2668 regions; Going from a computed imbalance of 0.18854544530991765 to a new imbalance of 0.016534851567460377. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023613042107609887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03886723784022791, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29644444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023613042107609887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,225 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv268694417:78 , srv573167608:79 , srv1675875015:80 , srv1425457541:81 , srv1546443560:81 , srv1955425326:81 , srv55801803:81 , srv1161805877:82 , srv1906260288:82 , srv1926098467:82 , srv2074465158:82 , srv599905775:82 , srv1133352723:83 , srv1294126017:83 , srv132287426:83 , srv301039938:83 , srv33601688:83 , srv360434320:83 , srv412632992:83 , srv666866736:83 , srv896138853:83 , srv1617502505:84 , srv16854906:84 , srv1948551866:84 , srv2037068909:84 , srv446042689:84 , srv516643137:84 , srv620199996:84 , srv831870762:84 , srv850812151:84 , srv888528300:84 , srv909234618:84 , srv106610178:85 , srv1195740816:85 , srv1542996779:85 , srv1555715886:85 , srv1611931384:85 , srv2070528577:85 , srv305378693:85 , srv383322525:85 , srv483326812:85 , srv529885738:85 , srv719016074:85 , srv770472355:85 , srv775174070:85 , srv781471636:85 , srv1153084798:86 , srv1264809760:86 , srv1323085712:86 , srv1420395625:86 , srv1466955296:86 , srv1543047617:86 , srv1586439184:86 , srv1718829405:86 , srv324173231:86 , srv643326428:86 , srv770640459:86 , srv1214491791:87 , srv1363247492:87 , srv1450479750:87 , srv157205079:87 , srv1624749773:87 , srv1693327223:87 , srv1718003531:87 , srv1903268418:87 , srv1988610093:87 , srv206213443:87 , srv20694211:87 , srv334102726:87 , srv639129758:87 , srv928972931:87 , srv931502993:87 , srv1379545824:88 , srv1438414460:88 , srv1797120305:88 , srv1845922243:88 , srv214542821:88 , srv283110205:88 , srv334710303:88 , srv474497597:88 , srv627045543:88 , srv639170583:88 , srv83643112:88 , srv1121732440:89 , srv1214970818:89 , srv1382150480:89 , srv1642480319:89 , srv1782688234:89 , srv252486933:89 , srv303831264:89 , srv317197783:89 , srv348580709:89 , srv527307780:89 , srv635977096:89 , srv684856381:89 , srv1657681266:90 , srv1940770336:90 , srv407287955:90 , srv871194874:92 , srv125490865:519 }
2023-10-19 21:57:01,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:01,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:01,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:01,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:01,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:01,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:01,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:01,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:01,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:01,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:01,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:01,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023613042107609887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03886723784022791, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023613042107609887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,338 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 21:57:01,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01651616983216764, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023613042107609887); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03886723784022791, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023613042107609887); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 21:57:01,494 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 232 ms to try 5000 different iterations.  Found a solution that moves 980 regions; Going from a computed imbalance of 0.01651616983216764 to a new imbalance of 9.578762978924907E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007120714615156442); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01198577533267986); MoveCostFunction : (multiplier=7.0, imbalance=0.10888888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007120714615156442); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00625); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,495 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1382150480:82 , srv1438414460:83 , srv1718003531:83 , srv1797120305:83 , srv1845922243:83 , srv303831264:83 , srv407287955:83 , srv527307780:83 , srv83643112:83 , srv871194874:83 , srv1323085712:84 , srv1420395625:84 , srv1466955296:84 , srv20694211:84 , srv348580709:84 , srv360434320:84 , srv516643137:84 , srv55801803:84 , srv639129758:84 , srv639170583:84 , srv684856381:84 , srv770472355:84 , srv931502993:84 , srv1153084798:85 , srv1161805877:85 , srv1195740816:85 , srv1214970818:85 , srv1450479750:85 , srv1642480319:85 , srv1657681266:85 , srv16854906:85 , srv1718829405:85 , srv1782688234:85 , srv1940770336:85 , srv1948551866:85 , srv2037068909:85 , srv283110205:85 , srv301039938:85 , srv324173231:85 , srv334102726:85 , srv474497597:85 , srv483326812:85 , srv573167608:85 , srv627045543:85 , srv719016074:85 , srv770640459:85 , srv781471636:85 , srv106610178:86 , srv1121732440:86 , srv1214491791:86 , srv1264809760:86 , srv1363247492:86 , srv1379545824:86 , srv1546443560:86 , srv1586439184:86 , srv1617502505:86 , srv1693327223:86 , srv1903268418:86 , srv1906260288:86 , srv1955425326:86 , srv1988610093:86 , srv206213443:86 , srv2074465158:86 , srv214542821:86 , srv252486933:86 , srv268694417:86 , srv305378693:86 , srv317197783:86 , srv334710303:86 , srv412632992:86 , srv446042689:86 , srv635977096:86 , srv643326428:86 , srv775174070:86 , srv831870762:86 , srv850812151:86 , srv909234618:86 , srv1294126017:87 , srv132287426:87 , srv1542996779:87 , srv1543047617:87 , srv157205079:87 , srv1624749773:87 , srv1675875015:87 , srv2070528577:87 , srv33601688:87 , srv383322525:87 , srv529885738:87 , srv599905775:87 , srv620199996:87 , srv888528300:87 , srv896138853:87 , srv928972931:87 , srv1133352723:88 , srv1425457541:88 , srv1555715886:88 , srv1611931384:88 , srv1926098467:89 , srv666866736:89 , srv125490865:537 }
2023-10-19 21:57:01,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:01,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:01,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:01,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:01,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:01,642 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007120714615156442); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01198577533267986); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007120714615156442); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00625); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,642 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 21:57:01,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.51014191225744E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007120714615156442); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01198577533267986); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007120714615156442); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00625); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 21:57:01,900 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 361 ms to try 5000 different iterations.  Found a solution that moves 332 regions; Going from a computed imbalance of 9.51014191225744E-4 to a new imbalance of 7.45216085191182E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.03688888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:01,901 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv666866736:82 , srv1450479750:83 , srv157205079:83 , srv1657681266:83 , srv1940770336:83 , srv20694211:83 , srv2070528577:83 , srv643326428:83 , srv928972931:83 , srv106610178:84 , srv1121732440:84 , srv1133352723:84 , srv132287426:84 , srv1363247492:84 , srv1379545824:84 , srv1420395625:84 , srv1425457541:84 , srv1542996779:84 , srv1543047617:84 , srv1555715886:84 , srv1675875015:84 , srv16854906:84 , srv1906260288:84 , srv1988610093:84 , srv2037068909:84 , srv206213443:84 , srv214542821:84 , srv283110205:84 , srv317197783:84 , srv348580709:84 , srv360434320:84 , srv474497597:84 , srv527307780:84 , srv599905775:84 , srv620199996:84 , srv639170583:84 , srv770472355:84 , srv775174070:84 , srv871194874:84 , srv888528300:84 , srv896138853:84 , srv1195740816:85 , srv1214491791:85 , srv1264809760:85 , srv1294126017:85 , srv1382150480:85 , srv1438414460:85 , srv1466955296:85 , srv1546443560:85 , srv1586439184:85 , srv1611931384:85 , srv1617502505:85 , srv1624749773:85 , srv1642480319:85 , srv1693327223:85 , srv1718003531:85 , srv1782688234:85 , srv1797120305:85 , srv1903268418:85 , srv1926098467:85 , srv1948551866:85 , srv2074465158:85 , srv252486933:85 , srv268694417:85 , srv301039938:85 , srv303831264:85 , srv324173231:85 , srv334102726:85 , srv334710303:85 , srv33601688:85 , srv412632992:85 , srv446042689:85 , srv483326812:85 , srv516643137:85 , srv529885738:85 , srv573167608:85 , srv627045543:85 , srv635977096:85 , srv639129758:85 , srv770640459:85 , srv781471636:85 , srv831870762:85 , srv83643112:85 , srv909234618:85 , srv1153084798:86 , srv1161805877:86 , srv1214970818:86 , srv1323085712:86 , srv1718829405:86 , srv1845922243:86 , srv1955425326:86 , srv305378693:86 , srv383322525:86 , srv407287955:86 , srv55801803:86 , srv684856381:86 , srv719016074:86 , srv850812151:86 , srv931502993:86 , srv125490865:621 }
2023-10-19 21:57:01,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:01,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:01,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:01,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:01,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:01,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:01,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:01,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:02,012 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:02,012 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-19 21:57:02,013 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.219689483201626E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=6.666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-19 21:57:02,203 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 272 ms to try 5000 different iterations.  Found a solution that moves 41 regions; Going from a computed imbalance of 7.219689483201626E-5 to a new imbalance of 8.546852122049172E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.004555555555555556); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:02,203 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1214970818:83 , srv1718829405:83 , srv1926098467:83 , srv1955425326:83 , srv305378693:83 , srv383322525:83 , srv684856381:83 , srv719016074:83 , srv775174070:83 , srv850812151:83 , srv106610178:84 , srv1121732440:84 , srv1133352723:84 , srv1153084798:84 , srv1161805877:84 , srv1195740816:84 , srv1214491791:84 , srv1264809760:84 , srv1294126017:84 , srv132287426:84 , srv1323085712:84 , srv1363247492:84 , srv1379545824:84 , srv1382150480:84 , srv1425457541:84 , srv1438414460:84 , srv1466955296:84 , srv1542996779:84 , srv1543047617:84 , srv1546443560:84 , srv1555715886:84 , srv157205079:84 , srv1586439184:84 , srv1611931384:84 , srv1617502505:84 , srv1624749773:84 , srv1642480319:84 , srv1657681266:84 , srv1675875015:84 , srv16854906:84 , srv1693327223:84 , srv1718003531:84 , srv1782688234:84 , srv1797120305:84 , srv1845922243:84 , srv1903268418:84 , srv1906260288:84 , srv1948551866:84 , srv1988610093:84 , srv2037068909:84 , srv206213443:84 , srv20694211:84 , srv2070528577:84 , srv2074465158:84 , srv214542821:84 , srv252486933:84 , srv268694417:84 , srv283110205:84 , srv301039938:84 , srv303831264:84 , srv317197783:84 , srv334102726:84 , srv334710303:84 , srv33601688:84 , srv360434320:84 , srv407287955:84 , srv412632992:84 , srv446042689:84 , srv474497597:84 , srv483326812:84 , srv527307780:84 , srv529885738:84 , srv55801803:84 , srv573167608:84 , srv599905775:84 , srv620199996:84 , srv627045543:84 , srv635977096:84 , srv639129758:84 , srv643326428:84 , srv770640459:84 , srv781471636:84 , srv831870762:84 , srv83643112:84 , srv871194874:84 , srv888528300:84 , srv909234618:84 , srv928972931:84 , srv931502993:84 , srv1420395625:85 , srv1450479750:85 , srv1940770336:85 , srv324173231:85 , srv348580709:85 , srv516643137:85 , srv639170583:85 , srv666866736:85 , srv770472355:85 , srv896138853:85 , srv125490865:684 }
2023-10-19 21:57:02,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:02,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1611931384=27, srv775174070=89, srv529885738=74, srv635977096=80, srv1542996779=21, srv1450479750=19, srv1133352723=2, srv1294126017=10, srv1382150480=15, srv888528300=95, srv16854906=33, srv1466955296=20, srv283110205=55, srv1438414460=18, srv639170583=82, srv1323085712=12, srv909234618=97, srv1121732440=1, srv2070528577=50, srv106610178=0, srv1642480319=30, srv1617502505=28, srv474497597=70, srv1555715886=24, srv360434320=65, srv214542821=52, srv1153084798=3, srv770472355=87, srv770640459=88, srv317197783=59, srv1955425326=45, srv303831264=57, srv301039938=56, srv83643112=92, srv1214491791=6, srv268694417=54, srv383322525=66, srv573167608=76, srv33601688=63, srv639129758=81, srv666866736=84, srv931502993=99, srv305378693=58, srv1906260288=41, srv2037068909=47, srv55801803=75, srv483326812=71, srv252486933=53, srv1214970818=7, srv1425457541=17, srv1624749773=29, srv1264809760=9, srv684856381=85, srv850812151=93, srv157205079=25, srv132287426=11, srv1988610093=46, srv896138853=96, srv1948551866=44, srv831870762=91, srv20694211=49, srv1718829405=36, srv1845922243=39, srv324173231=60, srv928972931=98, srv1903268418=40, srv1675875015=32, srv719016074=86, srv1657681266=31, srv643326428=83, srv620199996=78, srv1926098467=42, srv527307780=73, srv334710303=62, srv407287955=67, srv334102726=61, srv206213443=48, srv1718003531=35, srv348580709=64, srv599905775=77, srv781471636=90, srv1782688234=37, srv871194874=94, srv2074465158=51, srv1797120305=38, srv1195740816=5, srv412632992=68, srv125490865=8, srv1546443560=23, srv1379545824=14, srv1420395625=16, srv1161805877=4, srv1940770336=43, srv516643137=72, srv1693327223=34, srv1363247492=13, srv1543047617=22, srv446042689=69, srv627045543=79, srv1586439184=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:02,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:02,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:02,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:02,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-19 21:57:02,223 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-19 21:57:02,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-19 21:57:02,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-19 21:57:02,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-19 21:57:02,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-19 21:57:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-19 21:57:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-19 21:57:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-19 21:57:02,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-19 21:57:02,320 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:02,321 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1214970818:83 , srv1718829405:83 , srv1926098467:83 , srv1955425326:83 , srv305378693:83 , srv383322525:83 , srv684856381:83 , srv719016074:83 , srv775174070:83 , srv850812151:83 , srv106610178:84 , srv1121732440:84 , srv1133352723:84 , srv1153084798:84 , srv1161805877:84 , srv1195740816:84 , srv1214491791:84 , srv1264809760:84 , srv1294126017:84 , srv132287426:84 , srv1323085712:84 , srv1363247492:84 , srv1379545824:84 , srv1382150480:84 , srv1425457541:84 , srv1438414460:84 , srv1466955296:84 , srv1542996779:84 , srv1543047617:84 , srv1546443560:84 , srv1555715886:84 , srv157205079:84 , srv1586439184:84 , srv1611931384:84 , srv1617502505:84 , srv1624749773:84 , srv1642480319:84 , srv1657681266:84 , srv1675875015:84 , srv16854906:84 , srv1693327223:84 , srv1718003531:84 , srv1782688234:84 , srv1797120305:84 , srv1845922243:84 , srv1903268418:84 , srv1906260288:84 , srv1948551866:84 , srv1988610093:84 , srv2037068909:84 , srv206213443:84 , srv20694211:84 , srv2070528577:84 , srv2074465158:84 , srv214542821:84 , srv252486933:84 , srv268694417:84 , srv283110205:84 , srv301039938:84 , srv303831264:84 , srv317197783:84 , srv334102726:84 , srv334710303:84 , srv33601688:84 , srv360434320:84 , srv407287955:84 , srv412632992:84 , srv446042689:84 , srv474497597:84 , srv483326812:84 , srv527307780:84 , srv529885738:84 , srv55801803:84 , srv573167608:84 , srv599905775:84 , srv620199996:84 , srv627045543:84 , srv635977096:84 , srv639129758:84 , srv643326428:84 , srv770640459:84 , srv781471636:84 , srv831870762:84 , srv83643112:84 , srv871194874:84 , srv888528300:84 , srv909234618:84 , srv928972931:84 , srv931502993:84 , srv1420395625:85 , srv1450479750:85 , srv1940770336:85 , srv324173231:85 , srv348580709:85 , srv516643137:85 , srv639170583:85 , srv666866736:85 , srv770472355:85 , srv896138853:85 , srv125490865:684 }
2023-10-19 21:57:02,341 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=266 (was 220) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=6713 (was 7257)
2023-10-19 21:57:02,348 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=266, ProcessCount=168, AvailableMemoryMB=6712
2023-10-19 21:57:02,348 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-19 21:57:02,349 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-19 21:57:02,355 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1034794896:84 , srv1050527394:84 , srv1051056845:84 , srv1099410980:84 , srv1106734300:84 , srv1155718166:84 , srv1236453111:84 , srv1257056771:84 , srv1352379820:84 , srv1388773259:84 , srv1431808314:84 , srv1531783282:84 , srv1534098018:84 , srv1595955815:84 , srv1668524970:84 , srv1798927303:84 , srv1808165490:84 , srv1852148402:84 , srv1996954070:84 , srv2112514874:84 , srv285125855:84 , srv299685358:84 , srv415092699:84 , srv545851397:84 , srv794055920:84 , srv889328663:84 , srv933054508:84 , srv960372023:84 , srv961398099:84 , srv1768874320:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-19 21:57:02,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:57:02,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034794896=0, srv1236453111=6, srv285125855=21, srv1099410980=3, srv1534098018=12, srv1996954070=19, srv1257056771=7, srv961398099=29, srv1852148402=18, srv1388773259=9, srv1051056845=2, srv1431808314=10, srv1050527394=1, srv1668524970=14, srv415092699=23, srv889328663=26, srv1531783282=11, srv794055920=25, srv1352379820=8, srv299685358=22, srv1595955815=13, srv1106734300=4, srv1808165490=17, srv933054508=27, srv1155718166=5, srv1768874320=15, srv2112514874=20, srv960372023=28, srv1798927303=16, srv545851397=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:57:02,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:57:02,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:57:02,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:57:02,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:57:02,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:57:02,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-19 21:57:02,381 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0691287292475806, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:57:02,381 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912410865632879, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0691287292475806, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-19 21:59:02,369 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9404516 different iterations.  Found a solution that moves 1837 regions; Going from a computed imbalance of 0.9912410865632879 to a new imbalance of 4.2876496417733574E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6803703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:59:02,375 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1034794896:90 , srv1050527394:90 , srv1051056845:90 , srv1099410980:90 , srv1106734300:90 , srv1155718166:90 , srv1236453111:90 , srv1257056771:90 , srv1352379820:90 , srv1388773259:90 , srv1431808314:90 , srv1531783282:90 , srv1534098018:90 , srv1595955815:90 , srv1668524970:90 , srv1768874320:90 , srv1798927303:90 , srv1808165490:90 , srv1852148402:90 , srv1996954070:90 , srv2112514874:90 , srv285125855:90 , srv299685358:90 , srv415092699:90 , srv545851397:90 , srv794055920:90 , srv889328663:90 , srv933054508:90 , srv960372023:90 , srv961398099:90 }
2023-10-19 21:59:02,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-19 21:59:02,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1034794896=0, srv1236453111=6, srv285125855=21, srv1099410980=3, srv1534098018=12, srv1996954070=19, srv1257056771=7, srv961398099=29, srv1852148402=18, srv1388773259=9, srv1051056845=2, srv1431808314=10, srv1050527394=1, srv1668524970=14, srv415092699=23, srv889328663=26, srv1531783282=11, srv794055920=25, srv1352379820=8, srv299685358=22, srv1595955815=13, srv1106734300=4, srv1808165490=17, srv933054508=27, srv1155718166=5, srv1768874320=15, srv2112514874=20, srv960372023=28, srv1798927303=16, srv545851397=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-19 21:59:02,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-19 21:59:02,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-19 21:59:02,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-19 21:59:02,394 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-19 21:59:02,394 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1034794896:90 , srv1050527394:90 , srv1051056845:90 , srv1099410980:90 , srv1106734300:90 , srv1155718166:90 , srv1236453111:90 , srv1257056771:90 , srv1352379820:90 , srv1388773259:90 , srv1431808314:90 , srv1531783282:90 , srv1534098018:90 , srv1595955815:90 , srv1668524970:90 , srv1768874320:90 , srv1798927303:90 , srv1808165490:90 , srv1852148402:90 , srv1996954070:90 , srv2112514874:90 , srv285125855:90 , srv299685358:90 , srv415092699:90 , srv545851397:90 , srv794055920:90 , srv889328663:90 , srv933054508:90 , srv960372023:90 , srv961398099:90 }
2023-10-19 21:59:02,405 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=233 (was 266), ProcessCount=166 (was 168), AvailableMemoryMB=6350 (was 6712)