2023-08-31 02:53:39,128 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-31 02:53:39,449 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-31 02:53:39,563 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-31 02:53:39,628 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 02:53:39,635 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-31 02:53:39,649 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=262, ProcessCount=171, AvailableMemoryMB=11014
2023-08-31 02:53:39,651 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 02:53:39,652 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-31 02:53:39,795 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1034040453:84 , srv1072014871:84 , srv1094449870:84 , srv1107216929:84 , srv1119982710:84 , srv1124296074:84 , srv1134138487:84 , srv1159887565:84 , srv1170396622:84 , srv1183849002:84 , srv119964172:84 , srv1206184583:84 , srv123111503:84 , srv1232787092:84 , srv1235463128:84 , srv1235475577:84 , srv1242475280:84 , srv1272870651:84 , srv1284009464:84 , srv1303832060:84 , srv1314618131:84 , srv135670952:84 , srv1372005559:84 , srv1394548246:84 , srv1434027026:84 , srv1436298953:84 , srv1436592946:84 , srv1457187270:84 , srv1469147021:84 , srv1482931631:84 , srv1500115009:84 , srv151585100:84 , srv1521315863:84 , srv15330734:84 , srv1544042591:84 , srv1570152334:84 , srv157889846:84 , srv1604137454:84 , srv1605791817:84 , srv1613294147:84 , srv1638645835:84 , srv1652967392:84 , srv167035051:84 , srv1687004616:84 , srv1706253862:84 , srv1710785456:84 , srv1736148885:84 , srv1747053244:84 , srv1757774377:84 , srv1759499949:84 , srv1770476275:84 , srv1771078104:84 , srv1771104672:84 , srv1805361243:84 , srv1844500610:84 , srv1876475251:84 , srv1883311822:84 , srv1918119971:84 , srv1930496709:84 , srv1978062736:84 , srv1989726020:84 , srv2046259140:84 , srv2080728152:84 , srv2106204241:84 , srv211464031:84 , srv213399571:84 , srv2142505537:84 , srv244031082:84 , srv24717250:84 , srv251192843:84 , srv269697440:84 , srv293722029:84 , srv37961778:84 , srv391555526:84 , srv404651452:84 , srv410458298:84 , srv425506568:84 , srv49868011:84 , srv507044913:84 , srv576338509:84 , srv590077662:84 , srv674105635:84 , srv702331312:84 , srv758699126:84 , srv775975770:84 , srv818153227:84 , srv826785837:84 , srv827036466:84 , srv866851759:84 , srv871653183:84 , srv897370858:84 , srv911322984:84 , srv91422362:84 , srv920146140:84 , srv927172457:84 , srv941699336:84 , srv97617390:84 , srv977481125:84 , srv978094314:84 , srv199797259:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-31 02:53:39,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:39,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv213399571=66, srv1736148885=46, srv826785837=87, srv1652967392=41, srv1747053244=47, srv1119982710=4, srv1284009464=18, srv2106204241=64, srv1124296074=5, srv1232787092=13, srv1521315863=32, srv2142505537=67, srv1930496709=58, srv1372005559=22, srv24717250=69, srv1394548246=23, srv1457187270=27, srv1638645835=40, srv1159887565=7, srv978094314=99, srv123111503=12, srv1094449870=2, srv775975770=85, srv1500115009=30, srv1570152334=35, srv251192843=70, srv211464031=65, srv97617390=97, srv1436592946=26, srv1272870651=17, srv1883311822=56, srv167035051=42, srv49868011=78, srv1759499949=49, srv590077662=81, srv1183849002=9, srv1771078104=51, srv244031082=68, srv1034040453=0, srv1434027026=24, srv911322984=92, srv507044913=79, srv576338509=80, srv1314618131=20, srv1710785456=45, srv1107216929=3, srv157889846=36, srv391555526=74, srv1687004616=43, srv1072014871=1, srv871653183=90, srv1544042591=34, srv1805361243=53, srv293722029=72, srv1978062736=59, srv702331312=83, srv1482931631=29, srv1844500610=54, srv1235475577=15, srv91422362=93, srv1170396622=8, srv15330734=33, srv920146140=94, srv1303832060=19, srv977481125=98, srv151585100=31, srv1989726020=60, srv818153227=86, srv827036466=88, srv1770476275=50, srv1206184583=11, srv1134138487=6, srv2080728152=63, srv866851759=89, srv758699126=84, srv119964172=10, srv1235463128=14, srv135670952=21, srv1771104672=52, srv1706253862=44, srv425506568=77, srv1605791817=38, srv1876475251=55, srv1757774377=48, srv199797259=61, srv927172457=95, srv1604137454=37, srv37961778=73, srv1613294147=39, srv404651452=75, srv1918119971=57, srv1242475280=16, srv1436298953=25, srv941699336=96, srv674105635=82, srv1469147021=28, srv269697440=71, srv2046259140=62, srv897370858=91, srv410458298=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:39,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:39,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:39,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:39,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 02:53:39,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 02:53:39,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 02:53:39,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 02:53:39,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 02:53:39,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 02:53:39,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 02:53:39,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 02:53:39,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 02:53:39,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 02:53:39,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 02:53:39,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 02:53:40,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:40,072 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 02:53:40,073 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 02:53:40,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 544 ms to try 5000 different iterations.  Found a solution that moves 4680 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1809699987990272. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038488073956582866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05308020739888285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.52, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038488073956582866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:40,484 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1436298953:82 , srv1034040453:83 , srv1521315863:83 , srv1805361243:83 , srv2046259140:83 , srv211464031:83 , srv244031082:83 , srv871653183:83 , srv978094314:83 , srv1107216929:84 , srv1119982710:84 , srv1170396622:84 , srv1242475280:84 , srv1272870651:84 , srv1394548246:84 , srv151585100:84 , srv1844500610:84 , srv1883311822:84 , srv2106204241:84 , srv24717250:84 , srv37961778:84 , srv410458298:84 , srv507044913:84 , srv775975770:84 , srv827036466:84 , srv1159887565:85 , srv119964172:85 , srv1206184583:85 , srv1232787092:85 , srv1613294147:85 , srv1771078104:85 , srv1876475251:85 , srv293722029:85 , srv920146140:85 , srv927172457:85 , srv97617390:85 , srv1124296074:86 , srv1284009464:86 , srv1303832060:86 , srv135670952:86 , srv1372005559:86 , srv1434027026:86 , srv1457187270:86 , srv1500115009:86 , srv1544042591:86 , srv157889846:86 , srv1652967392:86 , srv1687004616:86 , srv1706253862:86 , srv1710785456:86 , srv1989726020:86 , srv425506568:86 , srv702331312:86 , srv818153227:86 , srv826785837:86 , srv941699336:86 , srv1094449870:87 , srv1235463128:87 , srv1235475577:87 , srv1747053244:87 , srv1757774377:87 , srv1771104672:87 , srv1918119971:87 , srv2080728152:87 , srv213399571:87 , srv576338509:87 , srv866851759:87 , srv911322984:87 , srv123111503:88 , srv1436592946:88 , srv1570152334:88 , srv1604137454:88 , srv1605791817:88 , srv1638645835:88 , srv1930496709:88 , srv2142505537:88 , srv251192843:88 , srv758699126:88 , srv977481125:88 , srv1072014871:89 , srv1770476275:89 , srv674105635:89 , srv897370858:89 , srv91422362:89 , srv1183849002:90 , srv1314618131:90 , srv1469147021:90 , srv167035051:90 , srv1736148885:90 , srv391555526:90 , srv49868011:90 , srv1134138487:91 , srv1978062736:91 , srv269697440:91 , srv1759499949:92 , srv404651452:92 , srv590077662:92 , srv15330734:93 , srv1482931631:94 , srv199797259:432 }
2023-08-31 02:53:40,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:40,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv213399571=66, srv1736148885=46, srv826785837=87, srv1652967392=41, srv1747053244=47, srv1119982710=4, srv1284009464=18, srv2106204241=64, srv1124296074=5, srv1232787092=13, srv1521315863=32, srv2142505537=67, srv1930496709=58, srv1372005559=22, srv24717250=69, srv1394548246=23, srv1457187270=27, srv1638645835=40, srv1159887565=7, srv978094314=99, srv123111503=12, srv1094449870=2, srv775975770=85, srv1500115009=30, srv1570152334=35, srv251192843=70, srv211464031=65, srv97617390=97, srv1436592946=26, srv1272870651=17, srv1883311822=56, srv167035051=42, srv49868011=78, srv1759499949=49, srv590077662=81, srv1183849002=9, srv1771078104=51, srv244031082=68, srv1034040453=0, srv1434027026=24, srv911322984=92, srv507044913=79, srv576338509=80, srv1314618131=20, srv1710785456=45, srv1107216929=3, srv157889846=36, srv391555526=74, srv1687004616=43, srv1072014871=1, srv871653183=90, srv1544042591=34, srv1805361243=53, srv293722029=72, srv1978062736=59, srv702331312=83, srv1482931631=29, srv1844500610=54, srv1235475577=15, srv91422362=93, srv1170396622=8, srv15330734=33, srv920146140=94, srv1303832060=19, srv977481125=98, srv151585100=31, srv1989726020=60, srv818153227=86, srv827036466=88, srv1770476275=50, srv1206184583=11, srv1134138487=6, srv2080728152=63, srv866851759=89, srv758699126=84, srv119964172=10, srv1235463128=14, srv135670952=21, srv1771104672=52, srv1706253862=44, srv425506568=77, srv1605791817=38, srv1876475251=55, srv1757774377=48, srv199797259=61, srv927172457=95, srv1604137454=37, srv37961778=73, srv1613294147=39, srv404651452=75, srv1918119971=57, srv1242475280=16, srv1436298953=25, srv941699336=96, srv674105635=82, srv1469147021=28, srv269697440=71, srv2046259140=62, srv897370858=91, srv410458298=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:40,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:40,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:40,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 02:53:40,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 02:53:40,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 02:53:40,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 02:53:40,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 02:53:40,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 02:53:40,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 02:53:40,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 02:53:40,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 02:53:40,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 02:53:40,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 02:53:40,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 02:53:40,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 02:53:40,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 02:53:40,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 02:53:40,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 02:53:40,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 02:53:40,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 02:53:40,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 02:53:40,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 02:53:40,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 02:53:40,661 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038488073956582866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05308020739888285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038488073956582866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:40,661 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 02:53:40,662 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18093722873861867, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038488073956582866, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05308020739888285, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038488073956582866, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.176, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24508333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 02:53:40,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 438 ms to try 5000 different iterations.  Found a solution that moves 2618 regions; Going from a computed imbalance of 0.18093722873861867 to a new imbalance of 0.013336764310088404. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020704634276744235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03325580208948566, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29088888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020704634276744235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:40,998 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1482931631:80 , srv391555526:80 , srv1134138487:81 , srv1314618131:81 , srv1469147021:81 , srv167035051:81 , srv1736148885:81 , srv1759499949:81 , srv404651452:81 , srv1072014871:82 , srv1183849002:82 , srv15330734:82 , srv1604137454:82 , srv251192843:82 , srv269697440:82 , srv91422362:82 , srv1436592946:83 , srv1570152334:83 , srv1638645835:83 , srv1770476275:83 , srv1978062736:83 , srv674105635:83 , srv1235463128:84 , srv1710785456:84 , srv1918119971:84 , srv1989726020:84 , srv2080728152:84 , srv2142505537:84 , srv590077662:84 , srv897370858:84 , srv911322984:84 , srv977481125:84 , srv1094449870:85 , srv1124296074:85 , srv1235475577:85 , srv1544042591:85 , srv1605791817:85 , srv1652967392:85 , srv1747053244:85 , srv1771104672:85 , srv49868011:85 , srv576338509:85 , srv818153227:85 , srv866851759:85 , srv1159887565:86 , srv123111503:86 , srv135670952:86 , srv1394548246:86 , srv1434027026:86 , srv1687004616:86 , srv1706253862:86 , srv1930496709:86 , srv213399571:86 , srv293722029:86 , srv410458298:86 , srv507044913:86 , srv826785837:86 , srv827036466:86 , srv941699336:86 , srv1107216929:87 , srv1206184583:87 , srv1232787092:87 , srv1303832060:87 , srv1372005559:87 , srv1457187270:87 , srv1757774377:87 , srv1771078104:87 , srv1844500610:87 , srv425506568:87 , srv702331312:87 , srv920146140:87 , srv927172457:87 , srv97617390:87 , srv1034040453:88 , srv1119982710:88 , srv1242475280:88 , srv1272870651:88 , srv1284009464:88 , srv1500115009:88 , srv157889846:88 , srv1613294147:88 , srv1883311822:88 , srv2046259140:88 , srv2106204241:88 , srv211464031:88 , srv37961778:88 , srv758699126:88 , srv775975770:88 , srv1170396622:89 , srv151585100:89 , srv1521315863:89 , srv1876475251:89 , srv871653183:89 , srv119964172:90 , srv244031082:90 , srv24717250:90 , srv978094314:90 , srv1436298953:91 , srv1805361243:91 , srv199797259:526 }
2023-08-31 02:53:41,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:41,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv213399571=66, srv1736148885=46, srv826785837=87, srv1652967392=41, srv1747053244=47, srv1119982710=4, srv1284009464=18, srv2106204241=64, srv1124296074=5, srv1232787092=13, srv1521315863=32, srv2142505537=67, srv1930496709=58, srv1372005559=22, srv24717250=69, srv1394548246=23, srv1457187270=27, srv1638645835=40, srv1159887565=7, srv978094314=99, srv123111503=12, srv1094449870=2, srv775975770=85, srv1500115009=30, srv1570152334=35, srv251192843=70, srv211464031=65, srv97617390=97, srv1436592946=26, srv1272870651=17, srv1883311822=56, srv167035051=42, srv49868011=78, srv1759499949=49, srv590077662=81, srv1183849002=9, srv1771078104=51, srv244031082=68, srv1034040453=0, srv1434027026=24, srv911322984=92, srv507044913=79, srv576338509=80, srv1314618131=20, srv1710785456=45, srv1107216929=3, srv157889846=36, srv391555526=74, srv1687004616=43, srv1072014871=1, srv871653183=90, srv1544042591=34, srv1805361243=53, srv293722029=72, srv1978062736=59, srv702331312=83, srv1482931631=29, srv1844500610=54, srv1235475577=15, srv91422362=93, srv1170396622=8, srv15330734=33, srv920146140=94, srv1303832060=19, srv977481125=98, srv151585100=31, srv1989726020=60, srv818153227=86, srv827036466=88, srv1770476275=50, srv1206184583=11, srv1134138487=6, srv2080728152=63, srv866851759=89, srv758699126=84, srv119964172=10, srv1235463128=14, srv135670952=21, srv1771104672=52, srv1706253862=44, srv425506568=77, srv1605791817=38, srv1876475251=55, srv1757774377=48, srv199797259=61, srv927172457=95, srv1604137454=37, srv37961778=73, srv1613294147=39, srv404651452=75, srv1918119971=57, srv1242475280=16, srv1436298953=25, srv941699336=96, srv674105635=82, srv1469147021=28, srv269697440=71, srv2046259140=62, srv897370858=91, srv410458298=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:41,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:41,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:41,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:41,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:41,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:41,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 02:53:41,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 02:53:41,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 02:53:41,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 02:53:41,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 02:53:41,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 02:53:41,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 02:53:41,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 02:53:41,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 02:53:41,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 02:53:41,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 02:53:41,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 02:53:41,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020704634276744235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03325580208948566, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020704634276744235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:41,199 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 02:53:41,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013318432682278667, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020704634276744235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03325580208948566, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020704634276744235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 02:53:41,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 498 ms to try 5000 different iterations.  Found a solution that moves 1000 regions; Going from a computed imbalance of 0.013318432682278667 to a new imbalance of 6.363053563690964E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005765937968227551); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.1111111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005765937968227551); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:41,566 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv758699126:81 , srv119964172:82 , srv1482931631:82 , srv15330734:82 , srv49868011:82 , srv590077662:82 , srv123111503:83 , srv1284009464:83 , srv1500115009:83 , srv1805361243:83 , srv1978062736:83 , srv24717250:83 , srv1242475280:84 , srv1303832060:84 , srv135670952:84 , srv157889846:84 , srv1757774377:84 , srv1771078104:84 , srv1876475251:84 , srv1930496709:84 , srv269697440:84 , srv674105635:84 , srv978094314:84 , srv1134138487:85 , srv1170396622:85 , srv1232787092:85 , srv1235475577:85 , srv1372005559:85 , srv1436298953:85 , srv1457187270:85 , srv151585100:85 , srv1521315863:85 , srv1605791817:85 , srv1613294147:85 , srv1759499949:85 , srv1771104672:85 , srv2106204241:85 , srv213399571:85 , srv2142505537:85 , srv244031082:85 , srv404651452:85 , srv425506568:85 , srv576338509:85 , srv702331312:85 , srv826785837:85 , srv866851759:85 , srv871653183:85 , srv97617390:85 , srv977481125:85 , srv1034040453:86 , srv1072014871:86 , srv1094449870:86 , srv1107216929:86 , srv1119982710:86 , srv1183849002:86 , srv1206184583:86 , srv1272870651:86 , srv1434027026:86 , srv1436592946:86 , srv1652967392:86 , srv167035051:86 , srv1687004616:86 , srv1706253862:86 , srv1736148885:86 , srv1747053244:86 , srv1770476275:86 , srv1883311822:86 , srv2046259140:86 , srv2080728152:86 , srv211464031:86 , srv37961778:86 , srv775975770:86 , srv818153227:86 , srv897370858:86 , srv920146140:86 , srv927172457:86 , srv941699336:86 , srv1159887565:87 , srv1235463128:87 , srv1314618131:87 , srv1394548246:87 , srv1469147021:87 , srv1544042591:87 , srv1570152334:87 , srv1604137454:87 , srv1638645835:87 , srv1844500610:87 , srv1918119971:87 , srv1989726020:87 , srv293722029:87 , srv391555526:87 , srv410458298:87 , srv507044913:87 , srv911322984:87 , srv91422362:87 , srv1124296074:88 , srv1710785456:88 , srv251192843:88 , srv827036466:88 , srv199797259:551 }
2023-08-31 02:53:41,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:41,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv213399571=66, srv1736148885=46, srv826785837=87, srv1652967392=41, srv1747053244=47, srv1119982710=4, srv1284009464=18, srv2106204241=64, srv1124296074=5, srv1232787092=13, srv1521315863=32, srv2142505537=67, srv1930496709=58, srv1372005559=22, srv24717250=69, srv1394548246=23, srv1457187270=27, srv1638645835=40, srv1159887565=7, srv978094314=99, srv123111503=12, srv1094449870=2, srv775975770=85, srv1500115009=30, srv1570152334=35, srv251192843=70, srv211464031=65, srv97617390=97, srv1436592946=26, srv1272870651=17, srv1883311822=56, srv167035051=42, srv49868011=78, srv1759499949=49, srv590077662=81, srv1183849002=9, srv1771078104=51, srv244031082=68, srv1034040453=0, srv1434027026=24, srv911322984=92, srv507044913=79, srv576338509=80, srv1314618131=20, srv1710785456=45, srv1107216929=3, srv157889846=36, srv391555526=74, srv1687004616=43, srv1072014871=1, srv871653183=90, srv1544042591=34, srv1805361243=53, srv293722029=72, srv1978062736=59, srv702331312=83, srv1482931631=29, srv1844500610=54, srv1235475577=15, srv91422362=93, srv1170396622=8, srv15330734=33, srv920146140=94, srv1303832060=19, srv977481125=98, srv151585100=31, srv1989726020=60, srv818153227=86, srv827036466=88, srv1770476275=50, srv1206184583=11, srv1134138487=6, srv2080728152=63, srv866851759=89, srv758699126=84, srv119964172=10, srv1235463128=14, srv135670952=21, srv1771104672=52, srv1706253862=44, srv425506568=77, srv1605791817=38, srv1876475251=55, srv1757774377=48, srv199797259=61, srv927172457=95, srv1604137454=37, srv37961778=73, srv1613294147=39, srv404651452=75, srv1918119971=57, srv1242475280=16, srv1436298953=25, srv941699336=96, srv674105635=82, srv1469147021=28, srv269697440=71, srv2046259140=62, srv897370858=91, srv410458298=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:41,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:41,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:41,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:41,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 02:53:41,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 02:53:41,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 02:53:41,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 02:53:41,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 02:53:41,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 02:53:41,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 02:53:41,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005765937968227551); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005765937968227551); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:41,698 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 02:53:41,698 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.293032067091508E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005765937968227551); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008632670957261212); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005765937968227551); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 02:53:42,185 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 583 ms to try 5000 different iterations.  Found a solution that moves 215 regions; Going from a computed imbalance of 6.293032067091508E-4 to a new imbalance of 1.1186016076598145E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.02388888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:42,187 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1124296074:83 , srv1687004616:83 , srv1759499949:83 , srv590077662:83 , srv897370858:83 , srv1094449870:84 , srv1119982710:84 , srv1159887565:84 , srv1170396622:84 , srv1183849002:84 , srv1206184583:84 , srv1232787092:84 , srv1235463128:84 , srv1272870651:84 , srv1314618131:84 , srv1372005559:84 , srv1434027026:84 , srv1436298953:84 , srv1457187270:84 , srv1469147021:84 , srv151585100:84 , srv1544042591:84 , srv1570152334:84 , srv157889846:84 , srv1605791817:84 , srv1613294147:84 , srv1638645835:84 , srv1706253862:84 , srv1710785456:84 , srv1747053244:84 , srv1757774377:84 , srv1770476275:84 , srv1844500610:84 , srv1876475251:84 , srv1883311822:84 , srv1918119971:84 , srv1930496709:84 , srv2080728152:84 , srv213399571:84 , srv251192843:84 , srv293722029:84 , srv37961778:84 , srv404651452:84 , srv425506568:84 , srv702331312:84 , srv775975770:84 , srv827036466:84 , srv911322984:84 , srv91422362:84 , srv920146140:84 , srv927172457:84 , srv1034040453:85 , srv1072014871:85 , srv1134138487:85 , srv119964172:85 , srv123111503:85 , srv1235475577:85 , srv1284009464:85 , srv1303832060:85 , srv1394548246:85 , srv1436592946:85 , srv1521315863:85 , srv15330734:85 , srv1604137454:85 , srv1652967392:85 , srv167035051:85 , srv1736148885:85 , srv1771104672:85 , srv1805361243:85 , srv1978062736:85 , srv2046259140:85 , srv2106204241:85 , srv211464031:85 , srv2142505537:85 , srv244031082:85 , srv24717250:85 , srv269697440:85 , srv391555526:85 , srv410458298:85 , srv49868011:85 , srv507044913:85 , srv576338509:85 , srv758699126:85 , srv818153227:85 , srv826785837:85 , srv866851759:85 , srv871653183:85 , srv941699336:85 , srv97617390:85 , srv977481125:85 , srv978094314:85 , srv1107216929:86 , srv1242475280:86 , srv135670952:86 , srv1482931631:86 , srv1500115009:86 , srv1771078104:86 , srv1989726020:86 , srv674105635:86 , srv199797259:633 }
2023-08-31 02:53:42,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:42,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv213399571=66, srv1736148885=46, srv826785837=87, srv1652967392=41, srv1747053244=47, srv1119982710=4, srv1284009464=18, srv2106204241=64, srv1124296074=5, srv1232787092=13, srv1521315863=32, srv2142505537=67, srv1930496709=58, srv1372005559=22, srv24717250=69, srv1394548246=23, srv1457187270=27, srv1638645835=40, srv1159887565=7, srv978094314=99, srv123111503=12, srv1094449870=2, srv775975770=85, srv1500115009=30, srv1570152334=35, srv251192843=70, srv211464031=65, srv97617390=97, srv1436592946=26, srv1272870651=17, srv1883311822=56, srv167035051=42, srv49868011=78, srv1759499949=49, srv590077662=81, srv1183849002=9, srv1771078104=51, srv244031082=68, srv1034040453=0, srv1434027026=24, srv911322984=92, srv507044913=79, srv576338509=80, srv1314618131=20, srv1710785456=45, srv1107216929=3, srv157889846=36, srv391555526=74, srv1687004616=43, srv1072014871=1, srv871653183=90, srv1544042591=34, srv1805361243=53, srv293722029=72, srv1978062736=59, srv702331312=83, srv1482931631=29, srv1844500610=54, srv1235475577=15, srv91422362=93, srv1170396622=8, srv15330734=33, srv920146140=94, srv1303832060=19, srv977481125=98, srv151585100=31, srv1989726020=60, srv818153227=86, srv827036466=88, srv1770476275=50, srv1206184583=11, srv1134138487=6, srv2080728152=63, srv866851759=89, srv758699126=84, srv119964172=10, srv1235463128=14, srv135670952=21, srv1771104672=52, srv1706253862=44, srv425506568=77, srv1605791817=38, srv1876475251=55, srv1757774377=48, srv199797259=61, srv927172457=95, srv1604137454=37, srv37961778=73, srv1613294147=39, srv404651452=75, srv1918119971=57, srv1242475280=16, srv1436298953=25, srv941699336=96, srv674105635=82, srv1469147021=28, srv269697440=71, srv2046259140=62, srv897370858=91, srv410458298=76} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:42,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:42,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 02:53:42,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 02:53:42,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 02:53:42,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 02:53:42,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 02:53:42,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 02:53:42,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.68055389970984E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:42,321 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1124296074:83 , srv1687004616:83 , srv1759499949:83 , srv590077662:83 , srv897370858:83 , srv1094449870:84 , srv1119982710:84 , srv1159887565:84 , srv1170396622:84 , srv1183849002:84 , srv1206184583:84 , srv1232787092:84 , srv1235463128:84 , srv1272870651:84 , srv1314618131:84 , srv1372005559:84 , srv1434027026:84 , srv1436298953:84 , srv1457187270:84 , srv1469147021:84 , srv151585100:84 , srv1544042591:84 , srv1570152334:84 , srv157889846:84 , srv1605791817:84 , srv1613294147:84 , srv1638645835:84 , srv1706253862:84 , srv1710785456:84 , srv1747053244:84 , srv1757774377:84 , srv1770476275:84 , srv1844500610:84 , srv1876475251:84 , srv1883311822:84 , srv1918119971:84 , srv1930496709:84 , srv2080728152:84 , srv213399571:84 , srv251192843:84 , srv293722029:84 , srv37961778:84 , srv404651452:84 , srv425506568:84 , srv702331312:84 , srv775975770:84 , srv827036466:84 , srv911322984:84 , srv91422362:84 , srv920146140:84 , srv927172457:84 , srv1034040453:85 , srv1072014871:85 , srv1134138487:85 , srv119964172:85 , srv123111503:85 , srv1235475577:85 , srv1284009464:85 , srv1303832060:85 , srv1394548246:85 , srv1436592946:85 , srv1521315863:85 , srv15330734:85 , srv1604137454:85 , srv1652967392:85 , srv167035051:85 , srv1736148885:85 , srv1771104672:85 , srv1805361243:85 , srv1978062736:85 , srv2046259140:85 , srv2106204241:85 , srv211464031:85 , srv2142505537:85 , srv244031082:85 , srv24717250:85 , srv269697440:85 , srv391555526:85 , srv410458298:85 , srv49868011:85 , srv507044913:85 , srv576338509:85 , srv758699126:85 , srv818153227:85 , srv826785837:85 , srv866851759:85 , srv871653183:85 , srv941699336:85 , srv97617390:85 , srv977481125:85 , srv978094314:85 , srv1107216929:86 , srv1242475280:86 , srv135670952:86 , srv1482931631:86 , srv1500115009:86 , srv1771078104:86 , srv1989726020:86 , srv674105635:86 , srv199797259:633 }
2023-08-31 02:53:42,363 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=305 (was 262) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=10213 (was 11014)
2023-08-31 02:53:42,374 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=305, ProcessCount=171, AvailableMemoryMB=10211
2023-08-31 02:53:42,375 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 02:53:42,375 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-31 02:53:42,387 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1030585614:84 , srv1122331703:84 , srv1272326883:84 , srv136307506:84 , srv1450578443:84 , srv1500745857:84 , srv1579512241:84 , srv158938815:84 , srv1683362906:84 , srv1696733947:84 , srv1730944629:84 , srv1795961759:84 , srv1891308893:84 , srv1919474042:84 , srv1962321678:84 , srv2093144316:84 , srv222363578:84 , srv390336589:84 , srv439114820:84 , srv478077819:84 , srv512440687:84 , srv515773867:84 , srv516566933:84 , srv660720425:84 , srv688374373:84 , srv718357191:84 , srv726537048:84 , srv973294765:84 , srv981740487:84 , srv1834531416:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-31 02:53:42,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:53:42,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962321678=15, srv1272326883=2, srv439114820=19, srv726537048=27, srv1122331703=1, srv390336589=18, srv1891308893=13, srv973294765=28, srv1683362906=8, srv516566933=23, srv1450578443=4, srv1919474042=14, srv478077819=20, srv1795961759=11, srv222363578=17, srv981740487=29, srv1696733947=9, srv1579512241=6, srv688374373=25, srv512440687=21, srv718357191=26, srv515773867=22, srv1030585614=0, srv660720425=24, srv1500745857=5, srv136307506=3, srv2093144316=16, srv1730944629=10, srv1834531416=12, srv158938815=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:53:42,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:53:42,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:53:42,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:53:42,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:53:42,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:53:42,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:53:42,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:53:42,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-31 02:53:42,435 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1568242630590506, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:53:42,435 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912687191396394, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1568242630590506, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-31 02:55:42,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5564147 different iterations.  Found a solution that moves 1863 regions; Going from a computed imbalance of 0.9912687191396394 to a new imbalance of 4.3483349388262195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.69, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:55:42,421 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1030585614:90 , srv1122331703:90 , srv1272326883:90 , srv136307506:90 , srv1450578443:90 , srv1500745857:90 , srv1579512241:90 , srv158938815:90 , srv1683362906:90 , srv1696733947:90 , srv1730944629:90 , srv1795961759:90 , srv1834531416:90 , srv1891308893:90 , srv1919474042:90 , srv1962321678:90 , srv2093144316:90 , srv222363578:90 , srv390336589:90 , srv439114820:90 , srv478077819:90 , srv512440687:90 , srv515773867:90 , srv516566933:90 , srv660720425:90 , srv688374373:90 , srv718357191:90 , srv726537048:90 , srv973294765:90 , srv981740487:90 }
2023-08-31 02:55:42,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 02:55:42,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1962321678=15, srv1272326883=2, srv439114820=19, srv726537048=27, srv1122331703=1, srv390336589=18, srv1891308893=13, srv973294765=28, srv1683362906=8, srv516566933=23, srv1450578443=4, srv1919474042=14, srv478077819=20, srv1795961759=11, srv222363578=17, srv981740487=29, srv1696733947=9, srv1579512241=6, srv688374373=25, srv512440687=21, srv718357191=26, srv515773867=22, srv1030585614=0, srv660720425=24, srv1500745857=5, srv136307506=3, srv2093144316=16, srv1730944629=10, srv1834531416=12, srv158938815=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 02:55:42,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 02:55:42,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 02:55:42,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 02:55:42,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-31 02:55:42,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 02:55:42,446 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1030585614:90 , srv1122331703:90 , srv1272326883:90 , srv136307506:90 , srv1450578443:90 , srv1500745857:90 , srv1579512241:90 , srv158938815:90 , srv1683362906:90 , srv1696733947:90 , srv1730944629:90 , srv1795961759:90 , srv1834531416:90 , srv1891308893:90 , srv1919474042:90 , srv1962321678:90 , srv2093144316:90 , srv222363578:90 , srv390336589:90 , srv439114820:90 , srv478077819:90 , srv512440687:90 , srv515773867:90 , srv516566933:90 , srv660720425:90 , srv688374373:90 , srv718357191:90 , srv726537048:90 , srv973294765:90 , srv981740487:90 }
2023-08-31 02:55:42,469 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 305), ProcessCount=172 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=10129 (was 10211)