2023-08-27 22:53:10,973 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-27 22:53:11,287 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-27 22:53:11,401 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-27 22:53:11,470 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-27 22:53:11,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-27 22:53:11,495 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=309, ProcessCount=171, AvailableMemoryMB=13056
2023-08-27 22:53:11,497 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-27 22:53:11,497 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-27 22:53:11,649 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003190623:84 , srv1059544142:84 , srv1077863525:84 , srv1112489421:84 , srv1114245766:84 , srv1156754376:84 , srv1159316538:84 , srv1175349058:84 , srv1184227763:84 , srv1195129666:84 , srv1240434016:84 , srv1245878525:84 , srv1260945849:84 , srv1278709166:84 , srv1288158906:84 , srv1293328826:84 , srv1295652202:84 , srv1298328060:84 , srv1315273938:84 , srv1378056882:84 , srv1388185051:84 , srv1424612617:84 , srv1438533435:84 , srv1451489292:84 , srv1456055517:84 , srv1459583213:84 , srv1499363189:84 , srv1507485620:84 , srv151049271:84 , srv1534744320:84 , srv154048159:84 , srv1605316405:84 , srv1611136148:84 , srv1615324001:84 , srv1630699196:84 , srv1634413698:84 , srv1637260157:84 , srv1645255849:84 , srv1672667901:84 , srv1673097031:84 , srv1700549742:84 , srv1700817010:84 , srv1768736376:84 , srv1768738241:84 , srv1778220813:84 , srv178657338:84 , srv1825781453:84 , srv1872048222:84 , srv187430811:84 , srv1880542201:84 , srv1928799500:84 , srv193123009:84 , srv1931823960:84 , srv1940739124:84 , srv1945691436:84 , srv1968949431:84 , srv1970488263:84 , srv1970579912:84 , srv1978786760:84 , srv1986376482:84 , srv2088445265:84 , srv256137336:84 , srv256983768:84 , srv341709964:84 , srv409553852:84 , srv417189087:84 , srv419807732:84 , srv427743845:84 , srv431888456:84 , srv463714679:84 , srv492837174:84 , srv525432699:84 , srv548997441:84 , srv572312070:84 , srv580292232:84 , srv637415002:84 , srv64500098:84 , srv652147960:84 , srv670335367:84 , srv685880948:84 , srv69811295:84 , srv722233445:84 , srv730529950:84 , srv730618002:84 , srv731548592:84 , srv756268381:84 , srv761510558:84 , srv76667106:84 , srv800935094:84 , srv807167932:84 , srv82519451:84 , srv859577826:84 , srv876016619:84 , srv883912963:84 , srv903743750:84 , srv922868167:84 , srv927521584:84 , srv959906002:84 , srv963582632:84 , srv342543111:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-27 22:53:11,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:11,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:11,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:11,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:11,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:11,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:11,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:11,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:11,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:11,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:11,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:11,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:11,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:11,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:11,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:12,006 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:12,007 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-27 22:53:12,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-27 22:53:12,415 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 624 ms to try 5000 different iterations.  Found a solution that moves 4703 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17198901759678772. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04193505034499584, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057528600216528614, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5225555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04193505034499584, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16708333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23408333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:12,452 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1245878525:82 , srv178657338:82 , srv1159316538:83 , srv1378056882:83 , srv1534744320:83 , srv1630699196:83 , srv1700549742:83 , srv1825781453:83 , srv1880542201:83 , srv1978786760:83 , srv572312070:83 , srv580292232:83 , srv731548592:83 , srv1077863525:84 , srv1114245766:84 , srv1295652202:84 , srv1456055517:84 , srv1507485620:84 , srv1605316405:84 , srv1768736376:84 , srv1940739124:84 , srv1945691436:84 , srv417189087:84 , srv427743845:84 , srv722233445:84 , srv1156754376:85 , srv1278709166:85 , srv1388185051:85 , srv1673097031:85 , srv1768738241:85 , srv1778220813:85 , srv1928799500:85 , srv1968949431:85 , srv1986376482:85 , srv256137336:85 , srv256983768:85 , srv883912963:85 , srv959906002:85 , srv1175349058:86 , srv1184227763:86 , srv1288158906:86 , srv1293328826:86 , srv1424612617:86 , srv1438533435:86 , srv1459583213:86 , srv154048159:86 , srv1634413698:86 , srv1672667901:86 , srv1700817010:86 , srv463714679:86 , srv525432699:86 , srv548997441:86 , srv730529950:86 , srv82519451:86 , srv927521584:86 , srv1059544142:87 , srv1260945849:87 , srv1499363189:87 , srv1637260157:87 , srv193123009:87 , srv1970579912:87 , srv341709964:87 , srv409553852:87 , srv431888456:87 , srv69811295:87 , srv807167932:87 , srv876016619:87 , srv903743750:87 , srv922868167:87 , srv1112489421:88 , srv1315273938:88 , srv1451489292:88 , srv1615324001:88 , srv1645255849:88 , srv1931823960:88 , srv2088445265:88 , srv419807732:88 , srv652147960:88 , srv685880948:88 , srv756268381:88 , srv761510558:88 , srv800935094:88 , srv859577826:88 , srv963582632:88 , srv1003190623:89 , srv187430811:89 , srv1970488263:89 , srv492837174:89 , srv670335367:89 , srv1195129666:90 , srv1240434016:90 , srv151049271:90 , srv1611136148:90 , srv637415002:90 , srv64500098:90 , srv730618002:90 , srv76667106:90 , srv1298328060:91 , srv1872048222:91 , srv342543111:463 }
2023-08-27 22:53:12,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:12,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:12,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:12,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:12,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:12,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:12,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:12,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:12,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:12,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:12,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:12,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:12,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:12,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:12,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:12,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:12,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:12,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:12,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:12,686 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04193505034499584, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057528600216528614, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04193505034499584, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16708333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23408333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:12,686 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-27 22:53:12,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.171956086486937, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04193505034499584, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057528600216528614, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04193505034499584, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16708333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23408333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-27 22:53:13,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 564 ms to try 5000 different iterations.  Found a solution that moves 2562 regions; Going from a computed imbalance of 0.171956086486937 to a new imbalance of 0.014692983284862298. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023171099732766); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03199326528457237, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2846666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023171099732766); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:13,116 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1872048222:80 , srv1195129666:81 , srv1611136148:81 , srv730618002:81 , srv1298328060:82 , srv1970488263:82 , srv419807732:82 , srv492837174:82 , srv76667106:82 , srv1003190623:83 , srv151049271:83 , srv1645255849:83 , srv685880948:83 , srv756268381:83 , srv1059544142:84 , srv1315273938:84 , srv1499363189:84 , srv1672667901:84 , srv1700817010:84 , srv193123009:84 , srv409553852:84 , srv431888456:84 , srv637415002:84 , srv64500098:84 , srv670335367:84 , srv800935094:84 , srv807167932:84 , srv903743750:84 , srv922868167:84 , srv927521584:84 , srv963582632:84 , srv1156754376:85 , srv1184227763:85 , srv1293328826:85 , srv1451489292:85 , srv1615324001:85 , srv1637260157:85 , srv1673097031:85 , srv1968949431:85 , srv1986376482:85 , srv2088445265:85 , srv341709964:85 , srv463714679:85 , srv525432699:85 , srv548997441:85 , srv652147960:85 , srv69811295:85 , srv761510558:85 , srv1114245766:86 , srv1175349058:86 , srv1240434016:86 , srv1260945849:86 , srv1278709166:86 , srv1438533435:86 , srv1459583213:86 , srv1507485620:86 , srv154048159:86 , srv1605316405:86 , srv1634413698:86 , srv1768738241:86 , srv187430811:86 , srv1931823960:86 , srv730529950:86 , srv859577826:86 , srv1112489421:87 , srv1295652202:87 , srv1388185051:87 , srv1630699196:87 , srv1778220813:87 , srv1928799500:87 , srv1978786760:87 , srv256137336:87 , srv82519451:87 , srv876016619:87 , srv959906002:87 , srv1159316538:88 , srv1288158906:88 , srv1378056882:88 , srv1424612617:88 , srv1534744320:88 , srv1700549742:88 , srv1768736376:88 , srv1825781453:88 , srv1940739124:88 , srv1945691436:88 , srv1970579912:88 , srv417189087:88 , srv427743845:88 , srv1077863525:89 , srv1456055517:89 , srv178657338:89 , srv1880542201:89 , srv256983768:89 , srv580292232:89 , srv722233445:89 , srv731548592:89 , srv572312070:90 , srv883912963:90 , srv1245878525:93 , srv342543111:517 }
2023-08-27 22:53:13,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:13,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:13,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:13,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:13,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:13,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:13,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:13,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:13,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:13,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:13,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:13,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:13,324 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023171099732766); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03199326528457237, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023171099732766); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:13,325 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-27 22:53:13,325 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014675043777433517, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023171099732766); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03199326528457237, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023171099732766); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0485, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-27 22:53:13,623 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 436 ms to try 5000 different iterations.  Found a solution that moves 1050 regions; Going from a computed imbalance of 0.014675043777433517 to a new imbalance of 0.0011118296424975568. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00946768777787555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013566976477075405); MoveCostFunction : (multiplier=7.0, imbalance=0.11666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00946768777787555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:13,624 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv187430811:81 , srv1112489421:82 , srv1240434016:82 , srv1245878525:82 , srv1424612617:82 , srv859577826:82 , srv151049271:83 , srv1931823960:83 , srv1970579912:83 , srv256983768:83 , srv572312070:83 , srv637415002:83 , srv883912963:83 , srv1003190623:84 , srv1288158906:84 , srv1298328060:84 , srv1451489292:84 , srv1456055517:84 , srv1615324001:84 , srv1928799500:84 , srv1945691436:84 , srv2088445265:84 , srv427743845:84 , srv64500098:84 , srv652147960:84 , srv722233445:84 , srv731548592:84 , srv82519451:84 , srv876016619:84 , srv1077863525:85 , srv1175349058:85 , srv1195129666:85 , srv1260945849:85 , srv1278709166:85 , srv1293328826:85 , srv1388185051:85 , srv1438533435:85 , srv1459583213:85 , srv1499363189:85 , srv1611136148:85 , srv1637260157:85 , srv1768736376:85 , srv1778220813:85 , srv1872048222:85 , srv1940739124:85 , srv341709964:85 , srv417189087:85 , srv685880948:85 , srv730529950:85 , srv761510558:85 , srv76667106:85 , srv1156754376:86 , srv1159316538:86 , srv1184227763:86 , srv1295652202:86 , srv1315273938:86 , srv1378056882:86 , srv154048159:86 , srv1605316405:86 , srv1634413698:86 , srv1645255849:86 , srv1700549742:86 , srv178657338:86 , srv1880542201:86 , srv193123009:86 , srv1970488263:86 , srv1978786760:86 , srv1986376482:86 , srv256137336:86 , srv409553852:86 , srv492837174:86 , srv580292232:86 , srv670335367:86 , srv69811295:86 , srv730618002:86 , srv800935094:86 , srv903743750:86 , srv922868167:86 , srv959906002:86 , srv1059544142:87 , srv1114245766:87 , srv1534744320:87 , srv1630699196:87 , srv1672667901:87 , srv1673097031:87 , srv1768738241:87 , srv1825781453:87 , srv1968949431:87 , srv463714679:87 , srv525432699:87 , srv548997441:87 , srv756268381:87 , srv807167932:87 , srv927521584:87 , srv963582632:87 , srv1507485620:88 , srv1700817010:88 , srv419807732:88 , srv431888456:88 , srv342543111:562 }
2023-08-27 22:53:13,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:13,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:13,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:13,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:13,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:13,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:13,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:13,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:13,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:13,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:13,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:13,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00946768777787555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013566976477075405); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00946768777787555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:13,825 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-27 22:53:13,826 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001104477385354614, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00946768777787555); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.013566976477075405); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00946768777787555); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-27 22:53:14,168 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 464 ms to try 5000 different iterations.  Found a solution that moves 282 regions; Going from a computed imbalance of 0.001104477385354614 to a new imbalance of 5.0298718025663326E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03133333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:14,170 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv431888456:83 , srv670335367:83 , srv963582632:83 , srv1059544142:84 , srv1077863525:84 , srv1184227763:84 , srv1260945849:84 , srv1288158906:84 , srv1295652202:84 , srv1315273938:84 , srv1378056882:84 , srv1507485620:84 , srv1534744320:84 , srv154048159:84 , srv1634413698:84 , srv1637260157:84 , srv1700817010:84 , srv1768738241:84 , srv1825781453:84 , srv1970488263:84 , srv256137336:84 , srv419807732:84 , srv525432699:84 , srv548997441:84 , srv580292232:84 , srv64500098:84 , srv69811295:84 , srv756268381:84 , srv761510558:84 , srv800935094:84 , srv807167932:84 , srv959906002:84 , srv1112489421:85 , srv1114245766:85 , srv1159316538:85 , srv1175349058:85 , srv1195129666:85 , srv1240434016:85 , srv1245878525:85 , srv1298328060:85 , srv1388185051:85 , srv1438533435:85 , srv1451489292:85 , srv1456055517:85 , srv151049271:85 , srv1615324001:85 , srv1630699196:85 , srv1645255849:85 , srv1672667901:85 , srv1673097031:85 , srv1700549742:85 , srv1768736376:85 , srv1778220813:85 , srv178657338:85 , srv1880542201:85 , srv193123009:85 , srv1931823960:85 , srv1968949431:85 , srv1970579912:85 , srv1986376482:85 , srv2088445265:85 , srv341709964:85 , srv409553852:85 , srv417189087:85 , srv463714679:85 , srv492837174:85 , srv637415002:85 , srv652147960:85 , srv722233445:85 , srv730529950:85 , srv730618002:85 , srv76667106:85 , srv82519451:85 , srv876016619:85 , srv883912963:85 , srv903743750:85 , srv922868167:85 , srv927521584:85 , srv1003190623:86 , srv1156754376:86 , srv1278709166:86 , srv1293328826:86 , srv1424612617:86 , srv1459583213:86 , srv1499363189:86 , srv1605316405:86 , srv1611136148:86 , srv1872048222:86 , srv187430811:86 , srv1928799500:86 , srv1940739124:86 , srv1945691436:86 , srv1978786760:86 , srv256983768:86 , srv427743845:86 , srv572312070:86 , srv685880948:86 , srv731548592:86 , srv859577826:86 , srv342543111:599 }
2023-08-27 22:53:14,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:14,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:14,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:14,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:14,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:14,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:14,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:14,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:14,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:14,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:14,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:14,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:14,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:14,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:14,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:14,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:14,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:14,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:14,352 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:14,352 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-27 22:53:14,353 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.8324111821558666E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-27 22:53:14,664 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 29 regions; Going from a computed imbalance of 4.8324111821558666E-5 to a new imbalance of 7.27630163646787E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0032222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:14,665 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1240434016:83 , srv1459583213:83 , srv1880542201:83 , srv1940739124:83 , srv1970579912:83 , srv256983768:83 , srv463714679:83 , srv876016619:83 , srv883912963:83 , srv1003190623:84 , srv1059544142:84 , srv1077863525:84 , srv1112489421:84 , srv1114245766:84 , srv1156754376:84 , srv1159316538:84 , srv1175349058:84 , srv1245878525:84 , srv1260945849:84 , srv1278709166:84 , srv1288158906:84 , srv1293328826:84 , srv1298328060:84 , srv1315273938:84 , srv1388185051:84 , srv1424612617:84 , srv1438533435:84 , srv1451489292:84 , srv1456055517:84 , srv1499363189:84 , srv1507485620:84 , srv1534744320:84 , srv154048159:84 , srv1605316405:84 , srv1611136148:84 , srv1615324001:84 , srv1630699196:84 , srv1634413698:84 , srv1645255849:84 , srv1672667901:84 , srv1673097031:84 , srv1700549742:84 , srv1700817010:84 , srv1768736376:84 , srv1768738241:84 , srv1778220813:84 , srv178657338:84 , srv1825781453:84 , srv1872048222:84 , srv187430811:84 , srv1928799500:84 , srv193123009:84 , srv1931823960:84 , srv1945691436:84 , srv1968949431:84 , srv1978786760:84 , srv2088445265:84 , srv256137336:84 , srv341709964:84 , srv409553852:84 , srv417189087:84 , srv419807732:84 , srv427743845:84 , srv431888456:84 , srv492837174:84 , srv525432699:84 , srv548997441:84 , srv572312070:84 , srv580292232:84 , srv637415002:84 , srv64500098:84 , srv652147960:84 , srv670335367:84 , srv685880948:84 , srv69811295:84 , srv722233445:84 , srv730529950:84 , srv730618002:84 , srv731548592:84 , srv756268381:84 , srv761510558:84 , srv76667106:84 , srv800935094:84 , srv807167932:84 , srv82519451:84 , srv859577826:84 , srv903743750:84 , srv922868167:84 , srv927521584:84 , srv959906002:84 , srv963582632:84 , srv1184227763:85 , srv1195129666:85 , srv1295652202:85 , srv1378056882:85 , srv151049271:85 , srv1637260157:85 , srv1970488263:85 , srv1986376482:85 , srv342543111:685 }
2023-08-27 22:53:14,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:14,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154048159=30, srv1175349058=7, srv417189087=66, srv427743845=68, srv64500098=77, srv959906002=98, srv1388185051=20, srv342543111=64, srv730529950=83, srv1768736376=42, srv1872048222=47, srv1970488263=56, srv670335367=79, srv1156754376=5, srv1260945849=12, srv1240434016=10, srv1968949431=55, srv1459583213=25, srv876016619=93, srv1295652202=16, srv800935094=89, srv1003190623=0, srv1630699196=34, srv178657338=45, srv685880948=80, srv1928799500=50, srv1940739124=53, srv151049271=28, srv1288158906=14, srv1112489421=3, srv963582632=99, srv1059544142=1, srv1534744320=29, srv1945691436=54, srv1615324001=33, srv761510558=87, srv1700549742=40, srv1672667901=38, srv1700817010=41, srv1978786760=58, srv1507485620=27, srv419807732=67, srv193123009=51, srv525432699=72, srv82519451=91, srv731548592=85, srv572312070=74, srv1195129666=9, srv1245878525=11, srv1159316538=6, srv1378056882=19, srv431888456=69, srv922868167=96, srv1184227763=8, srv1880542201=49, srv1611136148=32, srv1278709166=13, srv1970579912=57, srv927521584=97, srv76667106=88, srv637415002=76, srv807167932=90, srv341709964=63, srv548997441=73, srv409553852=65, srv722233445=82, srv883912963=94, srv1451489292=23, srv652147960=78, srv1293328826=15, srv859577826=92, srv463714679=70, srv1768738241=43, srv1634413698=35, srv256983768=62, srv756268381=86, srv1424612617=21, srv1438533435=22, srv1315273938=18, srv256137336=61, srv2088445265=60, srv1645255849=37, srv1931823960=52, srv1605316405=31, srv1825781453=46, srv1986376482=59, srv730618002=84, srv492837174=71, srv580292232=75, srv903743750=95, srv1778220813=44, srv1077863525=2, srv69811295=81, srv1499363189=26, srv1114245766=4, srv187430811=48, srv1637260157=36, srv1456055517=24, srv1298328060=17, srv1673097031=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:14,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:14,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:14,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-27 22:53:14,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-27 22:53:14,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-27 22:53:14,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-27 22:53:14,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-27 22:53:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-27 22:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-27 22:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-27 22:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-27 22:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-27 22:53:14,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-27 22:53:14,831 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:14,832 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1240434016:83 , srv1459583213:83 , srv1880542201:83 , srv1940739124:83 , srv1970579912:83 , srv256983768:83 , srv463714679:83 , srv876016619:83 , srv883912963:83 , srv1003190623:84 , srv1059544142:84 , srv1077863525:84 , srv1112489421:84 , srv1114245766:84 , srv1156754376:84 , srv1159316538:84 , srv1175349058:84 , srv1245878525:84 , srv1260945849:84 , srv1278709166:84 , srv1288158906:84 , srv1293328826:84 , srv1298328060:84 , srv1315273938:84 , srv1388185051:84 , srv1424612617:84 , srv1438533435:84 , srv1451489292:84 , srv1456055517:84 , srv1499363189:84 , srv1507485620:84 , srv1534744320:84 , srv154048159:84 , srv1605316405:84 , srv1611136148:84 , srv1615324001:84 , srv1630699196:84 , srv1634413698:84 , srv1645255849:84 , srv1672667901:84 , srv1673097031:84 , srv1700549742:84 , srv1700817010:84 , srv1768736376:84 , srv1768738241:84 , srv1778220813:84 , srv178657338:84 , srv1825781453:84 , srv1872048222:84 , srv187430811:84 , srv1928799500:84 , srv193123009:84 , srv1931823960:84 , srv1945691436:84 , srv1968949431:84 , srv1978786760:84 , srv2088445265:84 , srv256137336:84 , srv341709964:84 , srv409553852:84 , srv417189087:84 , srv419807732:84 , srv427743845:84 , srv431888456:84 , srv492837174:84 , srv525432699:84 , srv548997441:84 , srv572312070:84 , srv580292232:84 , srv637415002:84 , srv64500098:84 , srv652147960:84 , srv670335367:84 , srv685880948:84 , srv69811295:84 , srv722233445:84 , srv730529950:84 , srv730618002:84 , srv731548592:84 , srv756268381:84 , srv761510558:84 , srv76667106:84 , srv800935094:84 , srv807167932:84 , srv82519451:84 , srv859577826:84 , srv903743750:84 , srv922868167:84 , srv927521584:84 , srv959906002:84 , srv963582632:84 , srv1184227763:85 , srv1195129666:85 , srv1295652202:85 , srv1378056882:85 , srv151049271:85 , srv1637260157:85 , srv1970488263:85 , srv1986376482:85 , srv342543111:685 }
2023-08-27 22:53:14,869 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=348 (was 309) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=12193 (was 13056)
2023-08-27 22:53:14,881 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=348, ProcessCount=171, AvailableMemoryMB=12191
2023-08-27 22:53:14,882 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-27 22:53:14,882 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-27 22:53:14,895 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv103121041:84 , srv1040672409:84 , srv1099057634:84 , srv1154461177:84 , srv116604715:84 , srv1190085827:84 , srv1382576068:84 , srv1398994046:84 , srv1402505307:84 , srv1426649143:84 , srv1544709422:84 , srv1646452972:84 , srv1671733318:84 , srv1737034821:84 , srv1843673033:84 , srv1866972772:84 , srv1886605960:84 , srv1889928545:84 , srv2077829995:84 , srv2090654333:84 , srv2123337432:84 , srv283841960:84 , srv476953556:84 , srv497333785:84 , srv643442265:84 , srv651843604:84 , srv727756308:84 , srv777887413:84 , srv860829710:84 , srv172878730:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-27 22:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1154461177=3, srv1737034821=14, srv116604715=4, srv1398994046=7, srv476953556=23, srv1866972772=16, srv2123337432=21, srv1671733318=12, srv283841960=22, srv727756308=27, srv1426649143=9, srv497333785=24, srv1843673033=15, srv2077829995=19, srv860829710=29, srv1099057634=2, srv643442265=25, srv1402505307=8, srv1886605960=17, srv1382576068=6, srv1646452972=11, srv1040672409=1, srv2090654333=20, srv1190085827=5, srv172878730=13, srv777887413=28, srv103121041=0, srv1544709422=10, srv1889928545=18, srv651843604=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:53:14,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:53:14,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-27 22:53:14,931 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1092875392355226, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:53:14,932 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912537404731845, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1092875392355226, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-27 22:55:14,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5892483 different iterations.  Found a solution that moves 1849 regions; Going from a computed imbalance of 0.9912537404731845 to a new imbalance of 4.3156582404131396E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6848148148148148, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:55:14,923 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv103121041:90 , srv1040672409:90 , srv1099057634:90 , srv1154461177:90 , srv116604715:90 , srv1190085827:90 , srv1382576068:90 , srv1398994046:90 , srv1402505307:90 , srv1426649143:90 , srv1544709422:90 , srv1646452972:90 , srv1671733318:90 , srv172878730:90 , srv1737034821:90 , srv1843673033:90 , srv1866972772:90 , srv1886605960:90 , srv1889928545:90 , srv2077829995:90 , srv2090654333:90 , srv2123337432:90 , srv283841960:90 , srv476953556:90 , srv497333785:90 , srv643442265:90 , srv651843604:90 , srv727756308:90 , srv777887413:90 , srv860829710:90 }
2023-08-27 22:55:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-27 22:55:14,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1154461177=3, srv1737034821=14, srv116604715=4, srv1398994046=7, srv476953556=23, srv1866972772=16, srv2123337432=21, srv1671733318=12, srv283841960=22, srv727756308=27, srv1426649143=9, srv497333785=24, srv1843673033=15, srv2077829995=19, srv860829710=29, srv1099057634=2, srv643442265=25, srv1402505307=8, srv1886605960=17, srv1382576068=6, srv1646452972=11, srv1040672409=1, srv2090654333=20, srv1190085827=5, srv172878730=13, srv777887413=28, srv103121041=0, srv1544709422=10, srv1889928545=18, srv651843604=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-27 22:55:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-27 22:55:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-27 22:55:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-27 22:55:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-27 22:55:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-27 22:55:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-27 22:55:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-27 22:55:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-27 22:55:14,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-27 22:55:14,947 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv103121041:90 , srv1040672409:90 , srv1099057634:90 , srv1154461177:90 , srv116604715:90 , srv1190085827:90 , srv1382576068:90 , srv1398994046:90 , srv1402505307:90 , srv1426649143:90 , srv1544709422:90 , srv1646452972:90 , srv1671733318:90 , srv172878730:90 , srv1737034821:90 , srv1843673033:90 , srv1866972772:90 , srv1886605960:90 , srv1889928545:90 , srv2077829995:90 , srv2090654333:90 , srv2123337432:90 , srv283841960:90 , srv476953556:90 , srv497333785:90 , srv643442265:90 , srv651843604:90 , srv727756308:90 , srv777887413:90 , srv860829710:90 }
2023-08-27 22:55:14,962 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 348), ProcessCount=170 (was 171), AvailableMemoryMB=11948 (was 12191)