2023-08-11 08:53:09,061 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-11 08:53:09,388 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-11 08:53:09,510 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-11 08:53:09,573 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 08:53:09,580 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-11 08:53:09,597 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=266, ProcessCount=174, AvailableMemoryMB=4450
2023-08-11 08:53:09,600 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 08:53:09,600 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-11 08:53:09,757 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001518499:84 , srv1030933484:84 , srv1040144042:84 , srv1042073291:84 , srv1045678835:84 , srv1068326905:84 , srv1075827887:84 , srv1105596506:84 , srv1144582523:84 , srv1145240875:84 , srv1269502125:84 , srv1274339120:84 , srv1280160254:84 , srv1290954450:84 , srv1306258346:84 , srv1345903728:84 , srv1389947261:84 , srv1413917183:84 , srv1414077791:84 , srv1421963668:84 , srv1432085757:84 , srv1450414692:84 , srv145106118:84 , srv145928002:84 , srv1460043406:84 , srv1513087306:84 , srv153252350:84 , srv1547074513:84 , srv1570981649:84 , srv1578131223:84 , srv1584385543:84 , srv1620403907:84 , srv1631752156:84 , srv1655399969:84 , srv1668311666:84 , srv169870866:84 , srv1735663390:84 , srv175195628:84 , srv1756142631:84 , srv1781285309:84 , srv179281310:84 , srv1792896135:84 , srv1803594003:84 , srv1816411703:84 , srv1817549696:84 , srv1837172074:84 , srv1859019584:84 , srv1909747517:84 , srv1923824743:84 , srv1931850028:84 , srv1945964570:84 , srv1954326559:84 , srv1966622235:84 , srv1996031554:84 , srv2066104682:84 , srv2093552873:84 , srv2118742136:84 , srv227341469:84 , srv255648516:84 , srv268269:84 , srv275161198:84 , srv29155046:84 , srv301824347:84 , srv309781074:84 , srv315083268:84 , srv385951102:84 , srv419298230:84 , srv453418036:84 , srv465145250:84 , srv515300666:84 , srv526904986:84 , srv535385180:84 , srv566211015:84 , srv572562862:84 , srv594744550:84 , srv638675463:84 , srv649644612:84 , srv65884610:84 , srv689586903:84 , srv703521579:84 , srv74001468:84 , srv751970661:84 , srv756863575:84 , srv80566415:84 , srv814277558:84 , srv815437435:84 , srv829656124:84 , srv839197180:84 , srv862555720:84 , srv880214785:84 , srv881483330:84 , srv90180589:84 , srv935134823:84 , srv952485587:84 , srv972423329:84 , srv974126650:84 , srv97465209:84 , srv982532925:84 , srv992063536:84 , srv572290357:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-11 08:53:09,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:09,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv814277558=85, srv1432085757=20, srv1269502125=10, srv1290954450=13, srv992063536=99, srv526904986=70, srv465145250=68, srv1144582523=8, srv952485587=94, srv255648516=58, srv1578131223=29, srv535385180=71, srv97465209=97, srv1345903728=15, srv1655399969=33, srv881483330=91, srv90180589=92, srv2066104682=54, srv1931850028=49, srv1668311666=34, srv1996031554=53, srv935134823=93, srv1413917183=17, srv179281310=40, srv175195628=37, srv982532925=98, srv1045678835=4, srv301824347=62, srv972423329=95, srv1513087306=25, srv594744550=75, srv2093552873=55, srv315083268=64, srv515300666=69, srv1105596506=7, srv1030933484=1, srv1547074513=27, srv751970661=82, srv1460043406=24, srv1756142631=38, srv1450414692=21, srv1306258346=14, srv974126650=96, srv1803594003=42, srv145928002=23, srv1909747517=47, srv65884610=78, srv1145240875=9, srv829656124=87, srv1068326905=5, srv1817549696=44, srv1414077791=18, srv1001518499=0, srv2118742136=56, srv1584385543=30, srv1631752156=32, srv1042073291=3, srv1735663390=36, srv703521579=80, srv80566415=84, srv880214785=90, srv1859019584=46, srv689586903=79, srv815437435=86, srv1792896135=41, srv309781074=63, srv29155046=61, srv1040144042=2, srv1954326559=51, srv268269=59, srv419298230=66, srv1945964570=50, srv756863575=83, srv862555720=89, srv385951102=65, srv145106118=22, srv1966622235=52, srv1274339120=11, srv275161198=60, srv649644612=77, srv638675463=76, srv1389947261=16, srv572562862=74, srv1421963668=19, srv1570981649=28, srv572290357=73, srv169870866=35, srv227341469=57, srv1816411703=43, srv839197180=88, srv1923824743=48, srv1075827887=6, srv1781285309=39, srv1837172074=45, srv566211015=72, srv453418036=67, srv1620403907=31, srv74001468=81, srv1280160254=12, srv153252350=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:09,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:09,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:09,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:09,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:09,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:09,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:09,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 08:53:09,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 08:53:09,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 08:53:09,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 08:53:09,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 08:53:09,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 08:53:09,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 08:53:09,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 08:53:09,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 08:53:09,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 08:53:09,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 08:53:09,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 08:53:09,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 08:53:10,107 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:10,108 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 08:53:10,108 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 08:53:10,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 557 ms to try 5000 different iterations.  Found a solution that moves 4698 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18352351427603725. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04288510302371073, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052287730243401595, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.522, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04288510302371073, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:10,484 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv309781074:81 , srv572562862:81 , srv1068326905:82 , srv1144582523:82 , srv1274339120:82 , srv1547074513:82 , srv453418036:82 , srv703521579:82 , srv80566415:82 , srv829656124:82 , srv1045678835:83 , srv1269502125:83 , srv1306258346:83 , srv1413917183:83 , srv1620403907:83 , srv1735663390:83 , srv1954326559:83 , srv2066104682:83 , srv2093552873:83 , srv29155046:83 , srv751970661:83 , srv839197180:83 , srv1075827887:84 , srv1460043406:84 , srv1584385543:84 , srv1816411703:84 , srv1923824743:84 , srv649644612:84 , srv689586903:84 , srv97465209:84 , srv1042073291:85 , srv1105596506:85 , srv1421963668:85 , srv145928002:85 , srv1781285309:85 , srv1817549696:85 , srv1859019584:85 , srv1945964570:85 , srv526904986:85 , srv638675463:85 , srv65884610:85 , srv90180589:85 , srv935134823:85 , srv974126650:85 , srv992063536:85 , srv1145240875:86 , srv1389947261:86 , srv1432085757:86 , srv1966622235:86 , srv301824347:86 , srv465145250:86 , srv515300666:86 , srv535385180:86 , srv566211015:86 , srv594744550:86 , srv74001468:86 , srv815437435:86 , srv1040144042:87 , srv1513087306:87 , srv1631752156:87 , srv169870866:87 , srv179281310:87 , srv1837172074:87 , srv1909747517:87 , srv419298230:87 , srv972423329:87 , srv1001518499:88 , srv1290954450:88 , srv1570981649:88 , srv1655399969:88 , srv1756142631:88 , srv1803594003:88 , srv227341469:88 , srv268269:88 , srv275161198:88 , srv756863575:88 , srv881483330:88 , srv952485587:88 , srv982532925:88 , srv1280160254:89 , srv1345903728:89 , srv1414077791:89 , srv145106118:89 , srv1931850028:89 , srv1996031554:89 , srv2118742136:89 , srv255648516:89 , srv1030933484:90 , srv1578131223:90 , srv1668311666:90 , srv315083268:90 , srv814277558:90 , srv153252350:91 , srv175195628:91 , srv862555720:91 , srv880214785:91 , srv1450414692:94 , srv385951102:94 , srv1792896135:95 , srv572290357:471 }
2023-08-11 08:53:10,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:10,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv814277558=85, srv1432085757=20, srv1269502125=10, srv1290954450=13, srv992063536=99, srv526904986=70, srv465145250=68, srv1144582523=8, srv952485587=94, srv255648516=58, srv1578131223=29, srv535385180=71, srv97465209=97, srv1345903728=15, srv1655399969=33, srv881483330=91, srv90180589=92, srv2066104682=54, srv1931850028=49, srv1668311666=34, srv1996031554=53, srv935134823=93, srv1413917183=17, srv179281310=40, srv175195628=37, srv982532925=98, srv1045678835=4, srv301824347=62, srv972423329=95, srv1513087306=25, srv594744550=75, srv2093552873=55, srv315083268=64, srv515300666=69, srv1105596506=7, srv1030933484=1, srv1547074513=27, srv751970661=82, srv1460043406=24, srv1756142631=38, srv1450414692=21, srv1306258346=14, srv974126650=96, srv1803594003=42, srv145928002=23, srv1909747517=47, srv65884610=78, srv1145240875=9, srv829656124=87, srv1068326905=5, srv1817549696=44, srv1414077791=18, srv1001518499=0, srv2118742136=56, srv1584385543=30, srv1631752156=32, srv1042073291=3, srv1735663390=36, srv703521579=80, srv80566415=84, srv880214785=90, srv1859019584=46, srv689586903=79, srv815437435=86, srv1792896135=41, srv309781074=63, srv29155046=61, srv1040144042=2, srv1954326559=51, srv268269=59, srv419298230=66, srv1945964570=50, srv756863575=83, srv862555720=89, srv385951102=65, srv145106118=22, srv1966622235=52, srv1274339120=11, srv275161198=60, srv649644612=77, srv638675463=76, srv1389947261=16, srv572562862=74, srv1421963668=19, srv1570981649=28, srv572290357=73, srv169870866=35, srv227341469=57, srv1816411703=43, srv839197180=88, srv1923824743=48, srv1075827887=6, srv1781285309=39, srv1837172074=45, srv566211015=72, srv453418036=67, srv1620403907=31, srv74001468=81, srv1280160254=12, srv153252350=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:10,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:10,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:10,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 08:53:10,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 08:53:10,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 08:53:10,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 08:53:10,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 08:53:10,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 08:53:10,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 08:53:10,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 08:53:10,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 08:53:10,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 08:53:10,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 08:53:10,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 08:53:10,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 08:53:10,717 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04288510302371073, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052287730243401595, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04288510302371073, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:10,717 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 08:53:10,718 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18349061817693482, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04288510302371073, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.052287730243401595, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04288510302371073, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17883333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 08:53:11,056 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 492 ms to try 5000 different iterations.  Found a solution that moves 2655 regions; Going from a computed imbalance of 0.18349061817693482 to a new imbalance of 0.014761510066925514. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024189054609226476); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03264955230923183, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.295, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024189054609226476); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:11,068 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv385951102:75 , srv1450414692:77 , srv880214785:78 , srv1792896135:80 , srv862555720:80 , srv1414077791:81 , srv1668311666:81 , srv1931850028:81 , srv1996031554:81 , srv814277558:81 , srv153252350:82 , srv2118742136:82 , srv1345903728:83 , srv1578131223:83 , srv1631752156:83 , srv169870866:83 , srv1803594003:83 , srv275161198:83 , srv315083268:83 , srv515300666:83 , srv982532925:83 , srv1145240875:84 , srv1280160254:84 , srv1290954450:84 , srv1389947261:84 , srv1570981649:84 , srv1655399969:84 , srv175195628:84 , srv1909747517:84 , srv268269:84 , srv301824347:84 , srv74001468:84 , srv1001518499:85 , srv1030933484:85 , srv1432085757:85 , srv145106118:85 , srv1513087306:85 , srv1756142631:85 , srv179281310:85 , srv1837172074:85 , srv255648516:85 , srv419298230:85 , srv566211015:85 , srv756863575:85 , srv972423329:85 , srv1421963668:86 , srv1781285309:86 , srv1945964570:86 , srv1966622235:86 , srv227341469:86 , srv751970661:86 , srv90180589:86 , srv935134823:86 , srv952485587:86 , srv974126650:86 , srv992063536:86 , srv1040144042:87 , srv1045678835:87 , srv1075827887:87 , srv1460043406:87 , srv1620403907:87 , srv1817549696:87 , srv1859019584:87 , srv453418036:87 , srv465145250:87 , srv638675463:87 , srv65884610:87 , srv689586903:87 , srv815437435:87 , srv1042073291:88 , srv1068326905:88 , srv1144582523:88 , srv145928002:88 , srv1923824743:88 , srv2066104682:88 , srv2093552873:88 , srv535385180:88 , srv594744550:88 , srv881483330:88 , srv1105596506:89 , srv1735663390:89 , srv1816411703:89 , srv1954326559:89 , srv29155046:89 , srv309781074:89 , srv526904986:89 , srv572562862:89 , srv649644612:89 , srv829656124:89 , srv97465209:89 , srv1269502125:90 , srv1306258346:90 , srv1413917183:90 , srv1547074513:90 , srv1584385543:90 , srv703521579:90 , srv80566415:90 , srv839197180:90 , srv1274339120:91 , srv572290357:518 }
2023-08-11 08:53:11,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:11,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv814277558=85, srv1432085757=20, srv1269502125=10, srv1290954450=13, srv992063536=99, srv526904986=70, srv465145250=68, srv1144582523=8, srv952485587=94, srv255648516=58, srv1578131223=29, srv535385180=71, srv97465209=97, srv1345903728=15, srv1655399969=33, srv881483330=91, srv90180589=92, srv2066104682=54, srv1931850028=49, srv1668311666=34, srv1996031554=53, srv935134823=93, srv1413917183=17, srv179281310=40, srv175195628=37, srv982532925=98, srv1045678835=4, srv301824347=62, srv972423329=95, srv1513087306=25, srv594744550=75, srv2093552873=55, srv315083268=64, srv515300666=69, srv1105596506=7, srv1030933484=1, srv1547074513=27, srv751970661=82, srv1460043406=24, srv1756142631=38, srv1450414692=21, srv1306258346=14, srv974126650=96, srv1803594003=42, srv145928002=23, srv1909747517=47, srv65884610=78, srv1145240875=9, srv829656124=87, srv1068326905=5, srv1817549696=44, srv1414077791=18, srv1001518499=0, srv2118742136=56, srv1584385543=30, srv1631752156=32, srv1042073291=3, srv1735663390=36, srv703521579=80, srv80566415=84, srv880214785=90, srv1859019584=46, srv689586903=79, srv815437435=86, srv1792896135=41, srv309781074=63, srv29155046=61, srv1040144042=2, srv1954326559=51, srv268269=59, srv419298230=66, srv1945964570=50, srv756863575=83, srv862555720=89, srv385951102=65, srv145106118=22, srv1966622235=52, srv1274339120=11, srv275161198=60, srv649644612=77, srv638675463=76, srv1389947261=16, srv572562862=74, srv1421963668=19, srv1570981649=28, srv572290357=73, srv169870866=35, srv227341469=57, srv1816411703=43, srv839197180=88, srv1923824743=48, srv1075827887=6, srv1781285309=39, srv1837172074=45, srv566211015=72, srv453418036=67, srv1620403907=31, srv74001468=81, srv1280160254=12, srv153252350=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:11,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:11,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:11,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:11,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:11,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:11,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:11,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 08:53:11,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 08:53:11,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 08:53:11,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 08:53:11,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 08:53:11,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 08:53:11,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 08:53:11,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 08:53:11,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024189054609226476); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03264955230923183, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024189054609226476); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:11,279 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 08:53:11,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014742919359578359, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024189054609226476); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03264955230923183, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024189054609226476); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 08:53:11,676 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 549 ms to try 5000 different iterations.  Found a solution that moves 962 regions; Going from a computed imbalance of 0.014742919359578359 to a new imbalance of 7.923013494384466E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006444096124338012); MoveCostFunction : (multiplier=7.0, imbalance=0.1068888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:11,678 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1030933484:82 , srv175195628:82 , srv1792896135:82 , srv881483330:82 , srv1105596506:83 , srv145106118:83 , srv227341469:83 , srv535385180:83 , srv594744550:83 , srv97465209:83 , srv1040144042:84 , srv1306258346:84 , srv1413917183:84 , srv153252350:84 , srv1584385543:84 , srv1756142631:84 , srv255648516:84 , srv465145250:84 , srv526904986:84 , srv649644612:84 , srv756863575:84 , srv815437435:84 , srv952485587:84 , srv1001518499:85 , srv1042073291:85 , srv1269502125:85 , srv1274339120:85 , srv1280160254:85 , srv1432085757:85 , srv1450414692:85 , srv145928002:85 , srv1513087306:85 , srv1578131223:85 , srv1735663390:85 , srv1816411703:85 , srv1817549696:85 , srv1837172074:85 , srv1909747517:85 , srv1923824743:85 , srv1954326559:85 , srv1966622235:85 , srv29155046:85 , srv315083268:85 , srv419298230:85 , srv638675463:85 , srv80566415:85 , srv839197180:85 , srv972423329:85 , srv1290954450:86 , srv1345903728:86 , srv1421963668:86 , srv1547074513:86 , srv1570981649:86 , srv1655399969:86 , srv1781285309:86 , srv179281310:86 , srv1803594003:86 , srv1859019584:86 , srv1945964570:86 , srv2093552873:86 , srv268269:86 , srv275161198:86 , srv65884610:86 , srv703521579:86 , srv814277558:86 , srv862555720:86 , srv992063536:86 , srv1068326905:87 , srv1075827887:87 , srv1145240875:87 , srv1389947261:87 , srv1460043406:87 , srv1668311666:87 , srv169870866:87 , srv1931850028:87 , srv1996031554:87 , srv2066104682:87 , srv2118742136:87 , srv309781074:87 , srv566211015:87 , srv572562862:87 , srv689586903:87 , srv829656124:87 , srv90180589:87 , srv935134823:87 , srv974126650:87 , srv1045678835:88 , srv1144582523:88 , srv1414077791:88 , srv1620403907:88 , srv1631752156:88 , srv301824347:88 , srv515300666:88 , srv74001468:88 , srv751970661:88 , srv982532925:88 , srv385951102:89 , srv880214785:89 , srv453418036:90 , srv572290357:522 }
2023-08-11 08:53:11,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:11,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv814277558=85, srv1432085757=20, srv1269502125=10, srv1290954450=13, srv992063536=99, srv526904986=70, srv465145250=68, srv1144582523=8, srv952485587=94, srv255648516=58, srv1578131223=29, srv535385180=71, srv97465209=97, srv1345903728=15, srv1655399969=33, srv881483330=91, srv90180589=92, srv2066104682=54, srv1931850028=49, srv1668311666=34, srv1996031554=53, srv935134823=93, srv1413917183=17, srv179281310=40, srv175195628=37, srv982532925=98, srv1045678835=4, srv301824347=62, srv972423329=95, srv1513087306=25, srv594744550=75, srv2093552873=55, srv315083268=64, srv515300666=69, srv1105596506=7, srv1030933484=1, srv1547074513=27, srv751970661=82, srv1460043406=24, srv1756142631=38, srv1450414692=21, srv1306258346=14, srv974126650=96, srv1803594003=42, srv145928002=23, srv1909747517=47, srv65884610=78, srv1145240875=9, srv829656124=87, srv1068326905=5, srv1817549696=44, srv1414077791=18, srv1001518499=0, srv2118742136=56, srv1584385543=30, srv1631752156=32, srv1042073291=3, srv1735663390=36, srv703521579=80, srv80566415=84, srv880214785=90, srv1859019584=46, srv689586903=79, srv815437435=86, srv1792896135=41, srv309781074=63, srv29155046=61, srv1040144042=2, srv1954326559=51, srv268269=59, srv419298230=66, srv1945964570=50, srv756863575=83, srv862555720=89, srv385951102=65, srv145106118=22, srv1966622235=52, srv1274339120=11, srv275161198=60, srv649644612=77, srv638675463=76, srv1389947261=16, srv572562862=74, srv1421963668=19, srv1570981649=28, srv572290357=73, srv169870866=35, srv227341469=57, srv1816411703=43, srv839197180=88, srv1923824743=48, srv1075827887=6, srv1781285309=39, srv1837172074=45, srv566211015=72, srv453418036=67, srv1620403907=31, srv74001468=81, srv1280160254=12, srv153252350=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:11,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:11,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:11,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 08:53:11,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 08:53:11,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 08:53:11,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 08:53:11,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 08:53:11,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 08:53:11,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 08:53:11,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 08:53:11,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 08:53:11,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 08:53:11,843 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006444096124338012); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:11,843 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 08:53:11,844 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.855652814655788E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006444096124338012); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 08:53:12,256 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 526 ms to try 5000 different iterations.  Found a solution that moves 227 regions; Going from a computed imbalance of 7.855652814655788E-4 to a new imbalance of 2.091063752479798E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.025222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:12,257 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1269502125:83 , srv1290954450:83 , srv453418036:83 , srv935134823:83 , srv982532925:83 , srv1001518499:84 , srv1030933484:84 , srv1040144042:84 , srv1042073291:84 , srv1045678835:84 , srv1075827887:84 , srv1144582523:84 , srv1274339120:84 , srv1280160254:84 , srv1345903728:84 , srv145928002:84 , srv1460043406:84 , srv1547074513:84 , srv1570981649:84 , srv1578131223:84 , srv1620403907:84 , srv1655399969:84 , srv1668311666:84 , srv1792896135:84 , srv1816411703:84 , srv1817549696:84 , srv1837172074:84 , srv1859019584:84 , srv2066104682:84 , srv2118742136:84 , srv255648516:84 , srv268269:84 , srv301824347:84 , srv315083268:84 , srv385951102:84 , srv526904986:84 , srv566211015:84 , srv572562862:84 , srv649644612:84 , srv65884610:84 , srv689586903:84 , srv703521579:84 , srv74001468:84 , srv829656124:84 , srv839197180:84 , srv880214785:84 , srv881483330:84 , srv90180589:84 , srv952485587:84 , srv974126650:84 , srv1068326905:85 , srv1105596506:85 , srv1145240875:85 , srv1306258346:85 , srv1389947261:85 , srv1413917183:85 , srv1414077791:85 , srv1421963668:85 , srv1450414692:85 , srv145106118:85 , srv1513087306:85 , srv153252350:85 , srv1584385543:85 , srv1631752156:85 , srv169870866:85 , srv1735663390:85 , srv175195628:85 , srv1756142631:85 , srv1781285309:85 , srv179281310:85 , srv1909747517:85 , srv1923824743:85 , srv1931850028:85 , srv1945964570:85 , srv1966622235:85 , srv1996031554:85 , srv227341469:85 , srv275161198:85 , srv29155046:85 , srv309781074:85 , srv419298230:85 , srv465145250:85 , srv515300666:85 , srv535385180:85 , srv594744550:85 , srv751970661:85 , srv756863575:85 , srv80566415:85 , srv814277558:85 , srv815437435:85 , srv862555720:85 , srv992063536:85 , srv1432085757:86 , srv1803594003:86 , srv1954326559:86 , srv2093552873:86 , srv638675463:86 , srv972423329:86 , srv97465209:87 , srv572290357:632 }
2023-08-11 08:53:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:12,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv814277558=85, srv1432085757=20, srv1269502125=10, srv1290954450=13, srv992063536=99, srv526904986=70, srv465145250=68, srv1144582523=8, srv952485587=94, srv255648516=58, srv1578131223=29, srv535385180=71, srv97465209=97, srv1345903728=15, srv1655399969=33, srv881483330=91, srv90180589=92, srv2066104682=54, srv1931850028=49, srv1668311666=34, srv1996031554=53, srv935134823=93, srv1413917183=17, srv179281310=40, srv175195628=37, srv982532925=98, srv1045678835=4, srv301824347=62, srv972423329=95, srv1513087306=25, srv594744550=75, srv2093552873=55, srv315083268=64, srv515300666=69, srv1105596506=7, srv1030933484=1, srv1547074513=27, srv751970661=82, srv1460043406=24, srv1756142631=38, srv1450414692=21, srv1306258346=14, srv974126650=96, srv1803594003=42, srv145928002=23, srv1909747517=47, srv65884610=78, srv1145240875=9, srv829656124=87, srv1068326905=5, srv1817549696=44, srv1414077791=18, srv1001518499=0, srv2118742136=56, srv1584385543=30, srv1631752156=32, srv1042073291=3, srv1735663390=36, srv703521579=80, srv80566415=84, srv880214785=90, srv1859019584=46, srv689586903=79, srv815437435=86, srv1792896135=41, srv309781074=63, srv29155046=61, srv1040144042=2, srv1954326559=51, srv268269=59, srv419298230=66, srv1945964570=50, srv756863575=83, srv862555720=89, srv385951102=65, srv145106118=22, srv1966622235=52, srv1274339120=11, srv275161198=60, srv649644612=77, srv638675463=76, srv1389947261=16, srv572562862=74, srv1421963668=19, srv1570981649=28, srv572290357=73, srv169870866=35, srv227341469=57, srv1816411703=43, srv839197180=88, srv1923824743=48, srv1075827887=6, srv1781285309=39, srv1837172074=45, srv566211015=72, srv453418036=67, srv1620403907=31, srv74001468=81, srv1280160254=12, srv153252350=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:12,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 08:53:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 08:53:12,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 08:53:12,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 08:53:12,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 08:53:12,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 08:53:12,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 08:53:12,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 08:53:12,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 08:53:12,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.932114955199033E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:12,401 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1269502125:83 , srv1290954450:83 , srv453418036:83 , srv935134823:83 , srv982532925:83 , srv1001518499:84 , srv1030933484:84 , srv1040144042:84 , srv1042073291:84 , srv1045678835:84 , srv1075827887:84 , srv1144582523:84 , srv1274339120:84 , srv1280160254:84 , srv1345903728:84 , srv145928002:84 , srv1460043406:84 , srv1547074513:84 , srv1570981649:84 , srv1578131223:84 , srv1620403907:84 , srv1655399969:84 , srv1668311666:84 , srv1792896135:84 , srv1816411703:84 , srv1817549696:84 , srv1837172074:84 , srv1859019584:84 , srv2066104682:84 , srv2118742136:84 , srv255648516:84 , srv268269:84 , srv301824347:84 , srv315083268:84 , srv385951102:84 , srv526904986:84 , srv566211015:84 , srv572562862:84 , srv649644612:84 , srv65884610:84 , srv689586903:84 , srv703521579:84 , srv74001468:84 , srv829656124:84 , srv839197180:84 , srv880214785:84 , srv881483330:84 , srv90180589:84 , srv952485587:84 , srv974126650:84 , srv1068326905:85 , srv1105596506:85 , srv1145240875:85 , srv1306258346:85 , srv1389947261:85 , srv1413917183:85 , srv1414077791:85 , srv1421963668:85 , srv1450414692:85 , srv145106118:85 , srv1513087306:85 , srv153252350:85 , srv1584385543:85 , srv1631752156:85 , srv169870866:85 , srv1735663390:85 , srv175195628:85 , srv1756142631:85 , srv1781285309:85 , srv179281310:85 , srv1909747517:85 , srv1923824743:85 , srv1931850028:85 , srv1945964570:85 , srv1966622235:85 , srv1996031554:85 , srv227341469:85 , srv275161198:85 , srv29155046:85 , srv309781074:85 , srv419298230:85 , srv465145250:85 , srv515300666:85 , srv535385180:85 , srv594744550:85 , srv751970661:85 , srv756863575:85 , srv80566415:85 , srv814277558:85 , srv815437435:85 , srv862555720:85 , srv992063536:85 , srv1432085757:86 , srv1803594003:86 , srv1954326559:86 , srv2093552873:86 , srv638675463:86 , srv972423329:86 , srv97465209:87 , srv572290357:632 }
2023-08-11 08:53:12,434 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=316 (was 266) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=3665 (was 4450)
2023-08-11 08:53:12,445 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=316, ProcessCount=174, AvailableMemoryMB=3663
2023-08-11 08:53:12,446 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 08:53:12,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-11 08:53:12,457 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1036358221:84 , srv1082177304:84 , srv1115843957:84 , srv1275194599:84 , srv1290663074:84 , srv1339886130:84 , srv135018081:84 , srv1374903356:84 , srv1844431509:84 , srv1851172660:84 , srv1880170257:84 , srv1972286423:84 , srv2023479811:84 , srv2048469497:84 , srv2080993346:84 , srv20840189:84 , srv329965519:84 , srv464420763:84 , srv477159172:84 , srv546494876:84 , srv61979974:84 , srv672716005:84 , srv724492396:84 , srv833412431:84 , srv885116478:84 , srv8966372:84 , srv92982841:84 , srv986146869:84 , srv995879518:84 , srv1253265119:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-11 08:53:12,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:53:12,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1844431509=9, srv1851172660=10, srv477159172=19, srv2048469497=14, srv1275194599=4, srv1253265119=3, srv885116478=25, srv2080993346=15, srv20840189=16, srv464420763=18, srv8966372=26, srv546494876=20, srv833412431=24, srv986146869=28, srv1036358221=0, srv1082177304=1, srv724492396=23, srv1115843957=2, srv1880170257=11, srv92982841=27, srv1339886130=6, srv1374903356=8, srv1972286423=12, srv995879518=29, srv1290663074=5, srv135018081=7, srv2023479811=13, srv329965519=17, srv61979974=21, srv672716005=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:53:12,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:53:12,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:53:12,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:53:12,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:53:12,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-11 08:53:12,496 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1909007174405248, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:53:12,496 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912794565191452, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1909007174405248, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-11 08:55:12,478 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5890582 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912794565191452 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:55:12,486 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1036358221:90 , srv1082177304:90 , srv1115843957:90 , srv1253265119:90 , srv1275194599:90 , srv1290663074:90 , srv1339886130:90 , srv135018081:90 , srv1374903356:90 , srv1844431509:90 , srv1851172660:90 , srv1880170257:90 , srv1972286423:90 , srv2023479811:90 , srv2048469497:90 , srv2080993346:90 , srv20840189:90 , srv329965519:90 , srv464420763:90 , srv477159172:90 , srv546494876:90 , srv61979974:90 , srv672716005:90 , srv724492396:90 , srv833412431:90 , srv885116478:90 , srv8966372:90 , srv92982841:90 , srv986146869:90 , srv995879518:90 }
2023-08-11 08:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 08:55:12,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1844431509=9, srv1851172660=10, srv477159172=19, srv2048469497=14, srv1275194599=4, srv1253265119=3, srv885116478=25, srv2080993346=15, srv20840189=16, srv464420763=18, srv8966372=26, srv546494876=20, srv833412431=24, srv986146869=28, srv1036358221=0, srv1082177304=1, srv724492396=23, srv1115843957=2, srv1880170257=11, srv92982841=27, srv1339886130=6, srv1374903356=8, srv1972286423=12, srv995879518=29, srv1290663074=5, srv135018081=7, srv2023479811=13, srv329965519=17, srv61979974=21, srv672716005=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 08:55:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 08:55:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 08:55:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 08:55:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 08:55:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 08:55:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 08:55:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 08:55:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 08:55:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-11 08:55:12,512 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 08:55:12,513 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1036358221:90 , srv1082177304:90 , srv1115843957:90 , srv1253265119:90 , srv1275194599:90 , srv1290663074:90 , srv1339886130:90 , srv135018081:90 , srv1374903356:90 , srv1844431509:90 , srv1851172660:90 , srv1880170257:90 , srv1972286423:90 , srv2023479811:90 , srv2048469497:90 , srv2080993346:90 , srv20840189:90 , srv329965519:90 , srv464420763:90 , srv477159172:90 , srv546494876:90 , srv61979974:90 , srv672716005:90 , srv724492396:90 , srv833412431:90 , srv885116478:90 , srv8966372:90 , srv92982841:90 , srv986146869:90 , srv995879518:90 }
2023-08-11 08:55:12,528 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=182 (was 316), ProcessCount=171 (was 174), AvailableMemoryMB=3719 (was 3663) - AvailableMemoryMB LEAK? -