2023-08-16 04:53:03,691 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-16 04:53:03,927 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-16 04:53:04,027 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-16 04:53:04,076 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 04:53:04,081 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 04:53:04,092 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=480, ProcessCount=173, AvailableMemoryMB=3277
2023-08-16 04:53:04,094 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 04:53:04,095 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 04:53:04,222 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000665001:84 , srv1036205242:84 , srv1045726449:84 , srv1046274506:84 , srv105444118:84 , srv1056106969:84 , srv1058365379:84 , srv1065450090:84 , srv1066813519:84 , srv1081249376:84 , srv1093795014:84 , srv1127901509:84 , srv1189859616:84 , srv1199702151:84 , srv1206688766:84 , srv122426861:84 , srv122845513:84 , srv1231848920:84 , srv1234316684:84 , srv1271192055:84 , srv1276322785:84 , srv1289762276:84 , srv1297477624:84 , srv1303856954:84 , srv1338060231:84 , srv1349797147:84 , srv1358075107:84 , srv1380078476:84 , srv1394748585:84 , srv1407874177:84 , srv1413748731:84 , srv1466932501:84 , srv1484002108:84 , srv1515120947:84 , srv1533405405:84 , srv1536746417:84 , srv1576814557:84 , srv1577871059:84 , srv15921811:84 , srv1595311215:84 , srv1643184256:84 , srv1644934380:84 , srv1686781837:84 , srv1702528065:84 , srv1740033775:84 , srv1746313391:84 , srv1772068492:84 , srv1789888060:84 , srv182724475:84 , srv182939228:84 , srv1896927630:84 , srv1899577319:84 , srv1918086738:84 , srv1938295836:84 , srv1954302830:84 , srv1982544151:84 , srv2022033412:84 , srv202313508:84 , srv2036543246:84 , srv2047585656:84 , srv2140513367:84 , srv217492049:84 , srv219481913:84 , srv220126255:84 , srv224980360:84 , srv257855628:84 , srv289013403:84 , srv312634366:84 , srv320173636:84 , srv327105360:84 , srv369356850:84 , srv384669609:84 , srv389413255:84 , srv39363921:84 , srv401105166:84 , srv409175647:84 , srv412826176:84 , srv442567763:84 , srv525724830:84 , srv529412168:84 , srv534648865:84 , srv535515923:84 , srv555551184:84 , srv585318293:84 , srv603892929:84 , srv643551217:84 , srv645006753:84 , srv645279925:84 , srv650215824:84 , srv692288971:84 , srv700863601:84 , srv728171136:84 , srv778612444:84 , srv86291038:84 , srv869117675:84 , srv873731811:84 , srv878455298:84 , srv905083895:84 , srv938348688:84 , srv680314334:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-16 04:53:04,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:04,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv645006753=86, srv905083895=98, srv320173636=68, srv312634366=67, srv1407874177=29, srv529412168=79, srv1918086738=52, srv1000665001=0, srv1702528065=43, srv289013403=66, srv873731811=96, srv122845513=16, srv1297477624=22, srv555551184=82, srv1271192055=19, srv1644934380=41, srv585318293=83, srv1289762276=21, srv1466932501=31, srv2022033412=56, srv219481913=62, srv369356850=70, srv182939228=49, srv1066813519=8, srv1789888060=47, srv1380078476=27, srv202313508=57, srv1899577319=51, srv442567763=77, srv1740033775=44, srv1276322785=20, srv878455298=97, srv1234316684=18, srv1036205242=1, srv1056106969=5, srv401105166=74, srv1045726449=2, srv2140513367=60, srv534648865=80, srv86291038=94, srv1127901509=11, srv778612444=93, srv525724830=78, srv1046274506=3, srv869117675=95, srv1938295836=53, srv217492049=61, srv1772068492=46, srv1954302830=54, srv1199702151=13, srv105444118=4, srv224980360=64, srv1206688766=14, srv1231848920=17, srv1746313391=45, srv1394748585=28, srv1303856954=23, srv650215824=88, srv1093795014=10, srv1577871059=37, srv1982544151=55, srv1643184256=40, srv1338060231=24, srv1081249376=9, srv220126255=63, srv1576814557=36, srv412826176=76, srv1189859616=12, srv389413255=72, srv1515120947=33, srv2036543246=58, srv680314334=89, srv2047585656=59, srv1686781837=42, srv1065450090=7, srv645279925=87, srv1058365379=6, srv182724475=48, srv728171136=92, srv692288971=90, srv938348688=99, srv1533405405=34, srv1413748731=30, srv39363921=73, srv1484002108=32, srv384669609=71, srv15921811=38, srv1595311215=39, srv257855628=65, srv1536746417=35, srv1896927630=50, srv535515923=81, srv409175647=75, srv1358075107=26, srv700863601=91, srv122426861=15, srv327105360=69, srv1349797147=25, srv643551217=85, srv603892929=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:04,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:04,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:04,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 04:53:04,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 04:53:04,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 04:53:04,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 04:53:04,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 04:53:04,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 04:53:04,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 04:53:04,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 04:53:04,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 04:53:04,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 04:53:04,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 04:53:04,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 04:53:04,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:04,460 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 04:53:04,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 04:53:04,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 343 ms to try 5000 different iterations.  Found a solution that moves 4655 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18356674449582397. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039021578983779344, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05609235610151422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5172222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039021578983779344, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2504166666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:04,720 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1938295836:82 , srv1093795014:83 , srv1536746417:83 , srv2140513367:83 , srv1065450090:84 , srv122845513:84 , srv1338060231:84 , srv1576814557:84 , srv1896927630:84 , srv2022033412:84 , srv2036543246:84 , srv535515923:84 , srv555551184:84 , srv86291038:84 , srv869117675:84 , srv878455298:84 , srv105444118:85 , srv1066813519:85 , srv1081249376:85 , srv1127901509:85 , srv1199702151:85 , srv1413748731:85 , srv1515120947:85 , srv1533405405:85 , srv1644934380:85 , srv1746313391:85 , srv1772068492:85 , srv182724475:85 , srv1899577319:85 , srv1982544151:85 , srv202313508:85 , srv219481913:85 , srv224980360:85 , srv257855628:85 , srv312634366:85 , srv384669609:85 , srv389413255:85 , srv645006753:85 , srv938348688:85 , srv1000665001:86 , srv1036205242:86 , srv1046274506:86 , srv1058365379:86 , srv1206688766:86 , srv1234316684:86 , srv1466932501:86 , srv1643184256:86 , srv1954302830:86 , srv220126255:86 , srv289013403:86 , srv327105360:86 , srv39363921:86 , srv401105166:86 , srv412826176:86 , srv645279925:86 , srv650215824:86 , srv700863601:86 , srv1349797147:87 , srv1686781837:87 , srv1789888060:87 , srv217492049:87 , srv369356850:87 , srv534648865:87 , srv585318293:87 , srv692288971:87 , srv728171136:87 , srv873731811:87 , srv905083895:87 , srv1056106969:88 , srv1189859616:88 , srv122426861:88 , srv1231848920:88 , srv1271192055:88 , srv1276322785:88 , srv1303856954:88 , srv1380078476:88 , srv1407874177:88 , srv1740033775:88 , srv182939228:88 , srv409175647:88 , srv442567763:88 , srv525724830:88 , srv529412168:88 , srv603892929:88 , srv1289762276:89 , srv1358075107:89 , srv1595311215:89 , srv1702528065:89 , srv2047585656:89 , srv1045726449:90 , srv1394748585:90 , srv778612444:90 , srv1297477624:91 , srv1484002108:91 , srv1577871059:91 , srv15921811:91 , srv1918086738:92 , srv320173636:92 , srv643551217:93 , srv680314334:437 }
2023-08-16 04:53:04,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:04,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv645006753=86, srv905083895=98, srv320173636=68, srv312634366=67, srv1407874177=29, srv529412168=79, srv1918086738=52, srv1000665001=0, srv1702528065=43, srv289013403=66, srv873731811=96, srv122845513=16, srv1297477624=22, srv555551184=82, srv1271192055=19, srv1644934380=41, srv585318293=83, srv1289762276=21, srv1466932501=31, srv2022033412=56, srv219481913=62, srv369356850=70, srv182939228=49, srv1066813519=8, srv1789888060=47, srv1380078476=27, srv202313508=57, srv1899577319=51, srv442567763=77, srv1740033775=44, srv1276322785=20, srv878455298=97, srv1234316684=18, srv1036205242=1, srv1056106969=5, srv401105166=74, srv1045726449=2, srv2140513367=60, srv534648865=80, srv86291038=94, srv1127901509=11, srv778612444=93, srv525724830=78, srv1046274506=3, srv869117675=95, srv1938295836=53, srv217492049=61, srv1772068492=46, srv1954302830=54, srv1199702151=13, srv105444118=4, srv224980360=64, srv1206688766=14, srv1231848920=17, srv1746313391=45, srv1394748585=28, srv1303856954=23, srv650215824=88, srv1093795014=10, srv1577871059=37, srv1982544151=55, srv1643184256=40, srv1338060231=24, srv1081249376=9, srv220126255=63, srv1576814557=36, srv412826176=76, srv1189859616=12, srv389413255=72, srv1515120947=33, srv2036543246=58, srv680314334=89, srv2047585656=59, srv1686781837=42, srv1065450090=7, srv645279925=87, srv1058365379=6, srv182724475=48, srv728171136=92, srv692288971=90, srv938348688=99, srv1533405405=34, srv1413748731=30, srv39363921=73, srv1484002108=32, srv384669609=71, srv15921811=38, srv1595311215=39, srv257855628=65, srv1536746417=35, srv1896927630=50, srv535515923=81, srv409175647=75, srv1358075107=26, srv700863601=91, srv122426861=15, srv327105360=69, srv1349797147=25, srv643551217=85, srv603892929=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:04,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:04,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:04,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 04:53:04,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 04:53:04,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 04:53:04,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 04:53:04,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 04:53:04,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 04:53:04,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 04:53:04,845 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039021578983779344, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05609235610151422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039021578983779344, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2504166666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:04,845 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 04:53:04,845 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1835341494891569, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039021578983779344, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05609235610151422, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039021578983779344, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17833333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2504166666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 04:53:05,024 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 264 ms to try 5000 different iterations.  Found a solution that moves 2709 regions; Going from a computed imbalance of 0.1835341494891569 to a new imbalance of 0.014348529255684826. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020246023905338993); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030039256022027283, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.301, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020246023905338993); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,033 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1577871059:80 , srv320173636:80 , srv643551217:80 , srv1297477624:81 , srv1918086738:81 , srv1394748585:82 , srv1045726449:83 , srv1380078476:83 , srv15921811:83 , srv1595311215:83 , srv2047585656:83 , srv442567763:83 , srv603892929:83 , srv778612444:83 , srv1231848920:84 , srv1271192055:84 , srv1276322785:84 , srv1740033775:84 , srv1789888060:84 , srv1954302830:84 , srv525724830:84 , srv873731811:84 , srv905083895:84 , srv1056106969:85 , srv1066813519:85 , srv1189859616:85 , srv1199702151:85 , srv122426861:85 , srv1234316684:85 , srv1289762276:85 , srv1303856954:85 , srv1349797147:85 , srv1358075107:85 , srv1407874177:85 , srv1484002108:85 , srv1515120947:85 , srv1702528065:85 , srv1746313391:85 , srv182724475:85 , srv182939228:85 , srv1899577319:85 , srv217492049:85 , srv220126255:85 , srv327105360:85 , srv384669609:85 , srv39363921:85 , srv409175647:85 , srv585318293:85 , srv728171136:85 , srv1046274506:86 , srv1058365379:86 , srv1643184256:86 , srv369356850:86 , srv389413255:86 , srv401105166:86 , srv412826176:86 , srv529412168:86 , srv534648865:86 , srv650215824:86 , srv692288971:86 , srv700863601:86 , srv938348688:86 , srv1036205242:87 , srv105444118:87 , srv1206688766:87 , srv122845513:87 , srv1413748731:87 , srv1686781837:87 , srv1772068492:87 , srv1896927630:87 , srv1982544151:87 , srv202313508:87 , srv2036543246:87 , srv219481913:87 , srv224980360:87 , srv312634366:87 , srv535515923:87 , srv645279925:87 , srv86291038:87 , srv1000665001:88 , srv1081249376:88 , srv1338060231:88 , srv1466932501:88 , srv1533405405:88 , srv1576814557:88 , srv1644934380:88 , srv555551184:88 , srv645006753:88 , srv869117675:88 , srv878455298:88 , srv1065450090:89 , srv1536746417:89 , srv1938295836:89 , srv2022033412:89 , srv257855628:89 , srv289013403:89 , srv1093795014:90 , srv1127901509:90 , srv2140513367:90 , srv680314334:517 }
2023-08-16 04:53:05,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:05,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv645006753=86, srv905083895=98, srv320173636=68, srv312634366=67, srv1407874177=29, srv529412168=79, srv1918086738=52, srv1000665001=0, srv1702528065=43, srv289013403=66, srv873731811=96, srv122845513=16, srv1297477624=22, srv555551184=82, srv1271192055=19, srv1644934380=41, srv585318293=83, srv1289762276=21, srv1466932501=31, srv2022033412=56, srv219481913=62, srv369356850=70, srv182939228=49, srv1066813519=8, srv1789888060=47, srv1380078476=27, srv202313508=57, srv1899577319=51, srv442567763=77, srv1740033775=44, srv1276322785=20, srv878455298=97, srv1234316684=18, srv1036205242=1, srv1056106969=5, srv401105166=74, srv1045726449=2, srv2140513367=60, srv534648865=80, srv86291038=94, srv1127901509=11, srv778612444=93, srv525724830=78, srv1046274506=3, srv869117675=95, srv1938295836=53, srv217492049=61, srv1772068492=46, srv1954302830=54, srv1199702151=13, srv105444118=4, srv224980360=64, srv1206688766=14, srv1231848920=17, srv1746313391=45, srv1394748585=28, srv1303856954=23, srv650215824=88, srv1093795014=10, srv1577871059=37, srv1982544151=55, srv1643184256=40, srv1338060231=24, srv1081249376=9, srv220126255=63, srv1576814557=36, srv412826176=76, srv1189859616=12, srv389413255=72, srv1515120947=33, srv2036543246=58, srv680314334=89, srv2047585656=59, srv1686781837=42, srv1065450090=7, srv645279925=87, srv1058365379=6, srv182724475=48, srv728171136=92, srv692288971=90, srv938348688=99, srv1533405405=34, srv1413748731=30, srv39363921=73, srv1484002108=32, srv384669609=71, srv15921811=38, srv1595311215=39, srv257855628=65, srv1536746417=35, srv1896927630=50, srv535515923=81, srv409175647=75, srv1358075107=26, srv700863601=91, srv122426861=15, srv327105360=69, srv1349797147=25, srv643551217=85, srv603892929=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:05,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:05,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:05,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:05,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:05,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:05,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 04:53:05,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 04:53:05,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 04:53:05,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 04:53:05,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 04:53:05,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 04:53:05,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 04:53:05,146 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020246023905338993); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030039256022027283, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020246023905338993); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,146 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 04:53:05,147 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014329560432256035, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020246023905338993); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030039256022027283, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020246023905338993); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 04:53:05,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 236 ms to try 5000 different iterations.  Found a solution that moves 949 regions; Going from a computed imbalance of 0.014329560432256035 to a new imbalance of 4.095861106817364E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004103049699311091); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005124707431905383); MoveCostFunction : (multiplier=7.0, imbalance=0.10544444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004103049699311091); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,309 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1484002108:81 , srv1000665001:83 , srv1289762276:83 , srv257855628:83 , srv1045726449:84 , srv1081249376:84 , srv1093795014:84 , srv1127901509:84 , srv1303856954:84 , srv1358075107:84 , srv1407874177:84 , srv1466932501:84 , srv15921811:84 , srv1702528065:84 , srv1918086738:84 , srv289013403:84 , srv409175647:84 , srv529412168:84 , srv645006753:84 , srv692288971:84 , srv778612444:84 , srv1036205242:85 , srv1046274506:85 , srv1058365379:85 , srv1065450090:85 , srv1189859616:85 , srv1206688766:85 , srv122426861:85 , srv1231848920:85 , srv1297477624:85 , srv1338060231:85 , srv1349797147:85 , srv1394748585:85 , srv1413748731:85 , srv1533405405:85 , srv1536746417:85 , srv1595311215:85 , srv1644934380:85 , srv1686781837:85 , srv182939228:85 , srv2022033412:85 , srv2047585656:85 , srv2140513367:85 , srv217492049:85 , srv224980360:85 , srv312634366:85 , srv369356850:85 , srv525724830:85 , srv534648865:85 , srv585318293:85 , srv645279925:85 , srv700863601:85 , srv869117675:85 , srv878455298:85 , srv105444118:86 , srv1056106969:86 , srv1271192055:86 , srv1576814557:86 , srv1577871059:86 , srv1740033775:86 , srv1772068492:86 , srv1982544151:86 , srv202313508:86 , srv2036543246:86 , srv219481913:86 , srv220126255:86 , srv320173636:86 , srv327105360:86 , srv401105166:86 , srv412826176:86 , srv442567763:86 , srv555551184:86 , srv603892929:86 , srv643551217:86 , srv728171136:86 , srv86291038:86 , srv1199702151:87 , srv122845513:87 , srv1234316684:87 , srv1276322785:87 , srv1380078476:87 , srv1515120947:87 , srv1643184256:87 , srv1789888060:87 , srv182724475:87 , srv1896927630:87 , srv1899577319:87 , srv1938295836:87 , srv384669609:87 , srv39363921:87 , srv535515923:87 , srv650215824:87 , srv873731811:87 , srv905083895:87 , srv938348688:87 , srv1066813519:88 , srv1746313391:88 , srv1954302830:88 , srv389413255:88 , srv680314334:540 }
2023-08-16 04:53:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:05,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv645006753=86, srv905083895=98, srv320173636=68, srv312634366=67, srv1407874177=29, srv529412168=79, srv1918086738=52, srv1000665001=0, srv1702528065=43, srv289013403=66, srv873731811=96, srv122845513=16, srv1297477624=22, srv555551184=82, srv1271192055=19, srv1644934380=41, srv585318293=83, srv1289762276=21, srv1466932501=31, srv2022033412=56, srv219481913=62, srv369356850=70, srv182939228=49, srv1066813519=8, srv1789888060=47, srv1380078476=27, srv202313508=57, srv1899577319=51, srv442567763=77, srv1740033775=44, srv1276322785=20, srv878455298=97, srv1234316684=18, srv1036205242=1, srv1056106969=5, srv401105166=74, srv1045726449=2, srv2140513367=60, srv534648865=80, srv86291038=94, srv1127901509=11, srv778612444=93, srv525724830=78, srv1046274506=3, srv869117675=95, srv1938295836=53, srv217492049=61, srv1772068492=46, srv1954302830=54, srv1199702151=13, srv105444118=4, srv224980360=64, srv1206688766=14, srv1231848920=17, srv1746313391=45, srv1394748585=28, srv1303856954=23, srv650215824=88, srv1093795014=10, srv1577871059=37, srv1982544151=55, srv1643184256=40, srv1338060231=24, srv1081249376=9, srv220126255=63, srv1576814557=36, srv412826176=76, srv1189859616=12, srv389413255=72, srv1515120947=33, srv2036543246=58, srv680314334=89, srv2047585656=59, srv1686781837=42, srv1065450090=7, srv645279925=87, srv1058365379=6, srv182724475=48, srv728171136=92, srv692288971=90, srv938348688=99, srv1533405405=34, srv1413748731=30, srv39363921=73, srv1484002108=32, srv384669609=71, srv15921811=38, srv1595311215=39, srv257855628=65, srv1536746417=35, srv1896927630=50, srv535515923=81, srv409175647=75, srv1358075107=26, srv700863601=91, srv122426861=15, srv327105360=69, srv1349797147=25, srv643551217=85, srv603892929=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:05,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:05,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:05,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:05,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 04:53:05,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 04:53:05,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 04:53:05,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 04:53:05,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 04:53:05,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 04:53:05,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 04:53:05,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 04:53:05,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 04:53:05,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 04:53:05,430 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004103049699311091); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005124707431905383); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004103049699311091); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,431 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 04:53:05,431 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.0294107065444803E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004103049699311091); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005124707431905383); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004103049699311091); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 04:53:05,659 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 318 ms to try 5000 different iterations.  Found a solution that moves 170 regions; Going from a computed imbalance of 4.0294107065444803E-4 to a new imbalance of 1.724209573871481E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.01888888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,659 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1056106969:83 , srv1127901509:83 , srv1380078476:83 , srv1643184256:83 , srv1686781837:83 , srv202313508:83 , srv289013403:83 , srv535515923:83 , srv643551217:83 , srv650215824:83 , srv1036205242:84 , srv105444118:84 , srv1066813519:84 , srv1189859616:84 , srv1199702151:84 , srv1206688766:84 , srv122426861:84 , srv122845513:84 , srv1271192055:84 , srv1276322785:84 , srv1349797147:84 , srv1358075107:84 , srv1407874177:84 , srv1466932501:84 , srv1533405405:84 , srv1576814557:84 , srv15921811:84 , srv1644934380:84 , srv1702528065:84 , srv1740033775:84 , srv1746313391:84 , srv1772068492:84 , srv1789888060:84 , srv182939228:84 , srv1896927630:84 , srv1938295836:84 , srv1954302830:84 , srv1982544151:84 , srv2022033412:84 , srv219481913:84 , srv220126255:84 , srv320173636:84 , srv369356850:84 , srv389413255:84 , srv39363921:84 , srv401105166:84 , srv412826176:84 , srv442567763:84 , srv534648865:84 , srv555551184:84 , srv645279925:84 , srv873731811:84 , srv905083895:84 , srv938348688:84 , srv1000665001:85 , srv1045726449:85 , srv1046274506:85 , srv1058365379:85 , srv1065450090:85 , srv1081249376:85 , srv1093795014:85 , srv1231848920:85 , srv1234316684:85 , srv1289762276:85 , srv1297477624:85 , srv1303856954:85 , srv1338060231:85 , srv1394748585:85 , srv1413748731:85 , srv1484002108:85 , srv1515120947:85 , srv1536746417:85 , srv1577871059:85 , srv1595311215:85 , srv182724475:85 , srv1899577319:85 , srv1918086738:85 , srv2036543246:85 , srv2047585656:85 , srv2140513367:85 , srv217492049:85 , srv224980360:85 , srv257855628:85 , srv312634366:85 , srv327105360:85 , srv384669609:85 , srv409175647:85 , srv525724830:85 , srv529412168:85 , srv585318293:85 , srv603892929:85 , srv645006753:85 , srv700863601:85 , srv728171136:85 , srv778612444:85 , srv86291038:85 , srv869117675:85 , srv878455298:85 , srv692288971:86 , srv680314334:648 }
2023-08-16 04:53:05,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:05,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv645006753=86, srv905083895=98, srv320173636=68, srv312634366=67, srv1407874177=29, srv529412168=79, srv1918086738=52, srv1000665001=0, srv1702528065=43, srv289013403=66, srv873731811=96, srv122845513=16, srv1297477624=22, srv555551184=82, srv1271192055=19, srv1644934380=41, srv585318293=83, srv1289762276=21, srv1466932501=31, srv2022033412=56, srv219481913=62, srv369356850=70, srv182939228=49, srv1066813519=8, srv1789888060=47, srv1380078476=27, srv202313508=57, srv1899577319=51, srv442567763=77, srv1740033775=44, srv1276322785=20, srv878455298=97, srv1234316684=18, srv1036205242=1, srv1056106969=5, srv401105166=74, srv1045726449=2, srv2140513367=60, srv534648865=80, srv86291038=94, srv1127901509=11, srv778612444=93, srv525724830=78, srv1046274506=3, srv869117675=95, srv1938295836=53, srv217492049=61, srv1772068492=46, srv1954302830=54, srv1199702151=13, srv105444118=4, srv224980360=64, srv1206688766=14, srv1231848920=17, srv1746313391=45, srv1394748585=28, srv1303856954=23, srv650215824=88, srv1093795014=10, srv1577871059=37, srv1982544151=55, srv1643184256=40, srv1338060231=24, srv1081249376=9, srv220126255=63, srv1576814557=36, srv412826176=76, srv1189859616=12, srv389413255=72, srv1515120947=33, srv2036543246=58, srv680314334=89, srv2047585656=59, srv1686781837=42, srv1065450090=7, srv645279925=87, srv1058365379=6, srv182724475=48, srv728171136=92, srv692288971=90, srv938348688=99, srv1533405405=34, srv1413748731=30, srv39363921=73, srv1484002108=32, srv384669609=71, srv15921811=38, srv1595311215=39, srv257855628=65, srv1536746417=35, srv1896927630=50, srv535515923=81, srv409175647=75, srv1358075107=26, srv700863601=91, srv122426861=15, srv327105360=69, srv1349797147=25, srv643551217=85, srv603892929=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:05,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:05,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 04:53:05,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 04:53:05,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 04:53:05,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 04:53:05,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 04:53:05,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 04:53:05,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 04:53:05,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 04:53:05,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 04:53:05,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 04:53:05,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 04:53:05,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 04:53:05,760 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.605173029652406E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,761 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1056106969:83 , srv1127901509:83 , srv1380078476:83 , srv1643184256:83 , srv1686781837:83 , srv202313508:83 , srv289013403:83 , srv535515923:83 , srv643551217:83 , srv650215824:83 , srv1036205242:84 , srv105444118:84 , srv1066813519:84 , srv1189859616:84 , srv1199702151:84 , srv1206688766:84 , srv122426861:84 , srv122845513:84 , srv1271192055:84 , srv1276322785:84 , srv1349797147:84 , srv1358075107:84 , srv1407874177:84 , srv1466932501:84 , srv1533405405:84 , srv1576814557:84 , srv15921811:84 , srv1644934380:84 , srv1702528065:84 , srv1740033775:84 , srv1746313391:84 , srv1772068492:84 , srv1789888060:84 , srv182939228:84 , srv1896927630:84 , srv1938295836:84 , srv1954302830:84 , srv1982544151:84 , srv2022033412:84 , srv219481913:84 , srv220126255:84 , srv320173636:84 , srv369356850:84 , srv389413255:84 , srv39363921:84 , srv401105166:84 , srv412826176:84 , srv442567763:84 , srv534648865:84 , srv555551184:84 , srv645279925:84 , srv873731811:84 , srv905083895:84 , srv938348688:84 , srv1000665001:85 , srv1045726449:85 , srv1046274506:85 , srv1058365379:85 , srv1065450090:85 , srv1081249376:85 , srv1093795014:85 , srv1231848920:85 , srv1234316684:85 , srv1289762276:85 , srv1297477624:85 , srv1303856954:85 , srv1338060231:85 , srv1394748585:85 , srv1413748731:85 , srv1484002108:85 , srv1515120947:85 , srv1536746417:85 , srv1577871059:85 , srv1595311215:85 , srv182724475:85 , srv1899577319:85 , srv1918086738:85 , srv2036543246:85 , srv2047585656:85 , srv2140513367:85 , srv217492049:85 , srv224980360:85 , srv257855628:85 , srv312634366:85 , srv327105360:85 , srv384669609:85 , srv409175647:85 , srv525724830:85 , srv529412168:85 , srv585318293:85 , srv603892929:85 , srv645006753:85 , srv700863601:85 , srv728171136:85 , srv778612444:85 , srv86291038:85 , srv869117675:85 , srv878455298:85 , srv692288971:86 , srv680314334:648 }
2023-08-16 04:53:05,782 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=490 (was 480) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=2792 (was 3277)
2023-08-16 04:53:05,789 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=490, ProcessCount=175, AvailableMemoryMB=2792
2023-08-16 04:53:05,789 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 04:53:05,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-16 04:53:05,797 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003050181:84 , srv1043049869:84 , srv1046893938:84 , srv1056125815:84 , srv1259400003:84 , srv1288720998:84 , srv1320649459:84 , srv1348693825:84 , srv1382514393:84 , srv141054815:84 , srv1512221759:84 , srv1518929723:84 , srv1537275927:84 , srv1549519096:84 , srv1590641255:84 , srv1592003600:84 , srv1644427007:84 , srv1651800788:84 , srv1709123134:84 , srv1903712835:84 , srv1910330853:84 , srv2064457514:84 , srv2096432732:84 , srv223621436:84 , srv44049270:84 , srv482512420:84 , srv497290182:84 , srv527960582:84 , srv90614726:84 , srv774253331:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-16 04:53:05,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:53:05,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1320649459=6, srv1288720998=5, srv1348693825=7, srv1518929723=11, srv1592003600=15, srv1903712835=19, srv1537275927=12, srv1644427007=16, srv1382514393=8, srv1512221759=10, srv1709123134=18, srv527960582=27, srv2096432732=22, srv774253331=28, srv141054815=9, srv1910330853=20, srv1003050181=0, srv1259400003=4, srv2064457514=21, srv1549519096=13, srv482512420=25, srv90614726=29, srv223621436=23, srv1651800788=17, srv44049270=24, srv1590641255=14, srv1056125815=3, srv497290182=26, srv1046893938=2, srv1043049869=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:53:05,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:53:05,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 04:53:05,822 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.126855808875874, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:53:05,822 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912592761775824, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.126855808875874, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-16 04:55:05,810 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10216167 different iterations.  Found a solution that moves 1827 regions; Going from a computed imbalance of 0.9912592761775824 to a new imbalance of 4.264309142906872E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6766666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:55:05,816 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1003050181:90 , srv1043049869:90 , srv1046893938:90 , srv1056125815:90 , srv1259400003:90 , srv1288720998:90 , srv1320649459:90 , srv1348693825:90 , srv1382514393:90 , srv141054815:90 , srv1512221759:90 , srv1518929723:90 , srv1537275927:90 , srv1549519096:90 , srv1590641255:90 , srv1592003600:90 , srv1644427007:90 , srv1651800788:90 , srv1709123134:90 , srv1903712835:90 , srv1910330853:90 , srv2064457514:90 , srv2096432732:90 , srv223621436:90 , srv44049270:90 , srv482512420:90 , srv497290182:90 , srv527960582:90 , srv774253331:90 , srv90614726:90 }
2023-08-16 04:55:05,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 04:55:05,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1320649459=6, srv1288720998=5, srv1348693825=7, srv1518929723=11, srv1592003600=15, srv1903712835=19, srv1537275927=12, srv1644427007=16, srv1382514393=8, srv1512221759=10, srv1709123134=18, srv527960582=27, srv2096432732=22, srv774253331=28, srv141054815=9, srv1910330853=20, srv1003050181=0, srv1259400003=4, srv2064457514=21, srv1549519096=13, srv482512420=25, srv90614726=29, srv223621436=23, srv1651800788=17, srv44049270=24, srv1590641255=14, srv1056125815=3, srv497290182=26, srv1046893938=2, srv1043049869=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 04:55:05,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 04:55:05,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 04:55:05,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 04:55:05,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 04:55:05,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 04:55:05,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 04:55:05,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 04:55:05,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 04:55:05,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 04:55:05,839 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 04:55:05,840 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1003050181:90 , srv1043049869:90 , srv1046893938:90 , srv1056125815:90 , srv1259400003:90 , srv1288720998:90 , srv1320649459:90 , srv1348693825:90 , srv1382514393:90 , srv141054815:90 , srv1512221759:90 , srv1518929723:90 , srv1537275927:90 , srv1549519096:90 , srv1590641255:90 , srv1592003600:90 , srv1644427007:90 , srv1651800788:90 , srv1709123134:90 , srv1903712835:90 , srv1910330853:90 , srv2064457514:90 , srv2096432732:90 , srv223621436:90 , srv44049270:90 , srv482512420:90 , srv497290182:90 , srv527960582:90 , srv774253331:90 , srv90614726:90 }
2023-08-16 04:55:05,851 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=162 (was 490), ProcessCount=173 (was 175), AvailableMemoryMB=2584 (was 2792)