2023-03-08 23:54:12,865 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-08 23:54:13,174 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-08 23:54:13,276 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-08 23:54:13,333 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-08 23:54:13,339 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-08 23:54:13,352 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=222, ProcessCount=170, AvailableMemoryMB=8663
2023-03-08 23:54:13,354 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-08 23:54:13,354 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-08 23:54:13,478 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1016444074:84 , srv1052614715:84 , srv1061754300:84 , srv1111740851:84 , srv1142219635:84 , srv1148176017:84 , srv1194367741:84 , srv1230353945:84 , srv1256289722:84 , srv1256679487:84 , srv1263822128:84 , srv1282422630:84 , srv1299978469:84 , srv130965354:84 , srv131309242:84 , srv1329273609:84 , srv1331983984:84 , srv1373792323:84 , srv1411693917:84 , srv1428687616:84 , srv143527319:84 , srv1466055519:84 , srv1489700208:84 , srv1493021529:84 , srv1495866498:84 , srv1516033448:84 , srv1531822529:84 , srv1560990594:84 , srv1621936389:84 , srv1635535241:84 , srv1639557918:84 , srv1655056478:84 , srv1682025021:84 , srv1682707636:84 , srv1713506116:84 , srv1723943055:84 , srv1730907443:84 , srv1781043290:84 , srv1789648104:84 , srv1790983965:84 , srv1837715755:84 , srv1862500443:84 , srv1866078466:84 , srv1867285004:84 , srv1870030944:84 , srv1872608648:84 , srv1893121957:84 , srv1954347294:84 , srv1975022498:84 , srv1995356768:84 , srv2053845855:84 , srv2054680576:84 , srv2076504345:84 , srv2085583351:84 , srv2088278795:84 , srv305521630:84 , srv31464116:84 , srv318532246:84 , srv338800910:84 , srv344801525:84 , srv348447034:84 , srv356368898:84 , srv366198930:84 , srv367463610:84 , srv37707223:84 , srv380425974:84 , srv383607447:84 , srv396811548:84 , srv406810396:84 , srv421170616:84 , srv444044030:84 , srv453110845:84 , srv462936686:84 , srv467543906:84 , srv476086140:84 , srv490290401:84 , srv499443701:84 , srv50161266:84 , srv529392630:84 , srv538956242:84 , srv607506252:84 , srv613555182:84 , srv647636491:84 , srv654815810:84 , srv678967882:84 , srv685443373:84 , srv698233454:84 , srv737937951:84 , srv740284495:84 , srv740916019:84 , srv744852200:84 , srv769091316:84 , srv770741476:84 , srv809700605:84 , srv850555761:84 , srv894951630:84 , srv916028113:84 , srv931253118:84 , srv957242831:84 , srv649504141:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-08 23:54:13,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:13,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:13,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:13,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:13,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:13,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:13,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:13,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:13,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:13,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:13,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:13,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:13,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:13,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:13,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:13,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:13,710 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:13,711 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-08 23:54:13,711 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-08 23:54:14,022 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 439 ms to try 5000 different iterations.  Found a solution that moves 4670 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17980426148676928. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03854796853970999, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060725135549804146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5188888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03854796853970999, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:14,046 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1142219635:81 , srv396811548:81 , srv1493021529:82 , srv1789648104:82 , srv1872608648:82 , srv607506252:82 , srv1428687616:83 , srv1995356768:83 , srv37707223:83 , srv383607447:83 , srv740284495:83 , srv916028113:83 , srv1148176017:84 , srv1263822128:84 , srv1560990594:84 , srv1639557918:84 , srv1866078466:84 , srv1867285004:84 , srv647636491:84 , srv654815810:84 , srv740916019:84 , srv809700605:84 , srv1230353945:85 , srv130965354:85 , srv1331983984:85 , srv143527319:85 , srv1466055519:85 , srv1682025021:85 , srv1682707636:85 , srv2053845855:85 , srv2076504345:85 , srv2088278795:85 , srv318532246:85 , srv348447034:85 , srv476086140:85 , srv613555182:85 , srv737937951:85 , srv931253118:85 , srv1052614715:86 , srv1194367741:86 , srv1256289722:86 , srv131309242:86 , srv1373792323:86 , srv1621936389:86 , srv1870030944:86 , srv1893121957:86 , srv305521630:86 , srv338800910:86 , srv344801525:86 , srv366198930:86 , srv499443701:86 , srv529392630:86 , srv685443373:86 , srv698233454:86 , srv894951630:86 , srv1329273609:87 , srv1495866498:87 , srv1862500443:87 , srv1954347294:87 , srv380425974:87 , srv406810396:87 , srv444044030:87 , srv467543906:87 , srv50161266:87 , srv744852200:87 , srv770741476:87 , srv850555761:87 , srv1256679487:88 , srv1282422630:88 , srv1299978469:88 , srv1531822529:88 , srv1655056478:88 , srv1730907443:88 , srv356368898:88 , srv367463610:88 , srv421170616:88 , srv462936686:88 , srv678967882:88 , srv1111740851:89 , srv1411693917:89 , srv1489700208:89 , srv1723943055:89 , srv1781043290:89 , srv1837715755:89 , srv1975022498:89 , srv490290401:89 , srv538956242:89 , srv769091316:89 , srv1016444074:90 , srv1061754300:90 , srv1713506116:90 , srv31464116:90 , srv957242831:90 , srv1516033448:91 , srv2054680576:91 , srv453110845:91 , srv2085583351:92 , srv1635535241:95 , srv1790983965:106 , srv649504141:432 }
2023-03-08 23:54:14,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:14,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:14,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:14,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:14,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:14,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:14,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:14,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:14,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:14,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:14,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:14,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:14,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:14,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:14,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03854796853970999, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060725135549804146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03854796853970999, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:14,221 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-08 23:54:14,221 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17977156144785733, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03854796853970999, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.060725135549804146, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03854796853970999, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17525, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23925, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-08 23:54:14,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 2599 regions; Going from a computed imbalance of 0.17977156144785733 to a new imbalance of 0.013711847126928649. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136015204673266, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2887777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04133333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:14,533 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1790983965:66 , srv1635535241:77 , srv2054680576:79 , srv1411693917:80 , srv2085583351:80 , srv453110845:80 , srv1016444074:81 , srv1516033448:81 , srv490290401:81 , srv1061754300:82 , srv1299978469:82 , srv1975022498:82 , srv31464116:82 , srv957242831:82 , srv1111740851:83 , srv1489700208:83 , srv1723943055:83 , srv1781043290:83 , srv1837715755:83 , srv1954347294:83 , srv1282422630:84 , srv131309242:84 , srv143527319:84 , srv1495866498:84 , srv1531822529:84 , srv1730907443:84 , srv1862500443:84 , srv338800910:84 , srv380425974:84 , srv444044030:84 , srv462936686:84 , srv538956242:84 , srv1148176017:85 , srv1329273609:85 , srv1621936389:85 , srv1655056478:85 , srv1713506116:85 , srv1893121957:85 , srv356368898:85 , srv367463610:85 , srv467543906:85 , srv850555761:85 , srv1052614715:86 , srv1194367741:86 , srv1256679487:86 , srv130965354:86 , srv1373792323:86 , srv1560990594:86 , srv1682025021:86 , srv1867285004:86 , srv2088278795:86 , srv305521630:86 , srv318532246:86 , srv344801525:86 , srv348447034:86 , srv366198930:86 , srv421170616:86 , srv50161266:86 , srv613555182:86 , srv678967882:86 , srv698233454:86 , srv744852200:86 , srv769091316:86 , srv770741476:86 , srv894951630:86 , srv1466055519:87 , srv1639557918:87 , srv2076504345:87 , srv406810396:87 , srv476086140:87 , srv529392630:87 , srv685443373:87 , srv1230353945:88 , srv1263822128:88 , srv1331983984:88 , srv1682707636:88 , srv1995356768:88 , srv2053845855:88 , srv383607447:88 , srv499443701:88 , srv647636491:88 , srv737937951:88 , srv809700605:88 , srv1142219635:89 , srv1256289722:89 , srv1428687616:89 , srv1493021529:89 , srv1866078466:89 , srv37707223:89 , srv607506252:89 , srv654815810:89 , srv740284495:89 , srv916028113:89 , srv1789648104:90 , srv1870030944:90 , srv1872608648:90 , srv740916019:90 , srv396811548:91 , srv931253118:91 , srv649504141:542 }
2023-03-08 23:54:14,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:14,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:14,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:14,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:14,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:14,679 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136015204673266, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04133333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:14,679 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-08 23:54:14,679 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013693648539962449, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022497176666746645); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136015204673266, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022497176666746645); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04133333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-08 23:54:14,967 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 387 ms to try 5000 different iterations.  Found a solution that moves 963 regions; Going from a computed imbalance of 0.013693648539962449 to a new imbalance of 0.0010437689949068015. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006991702277396585); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011231669191044557); MoveCostFunction : (multiplier=7.0, imbalance=0.107, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006991702277396585); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:14,971 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1870030944:81 , srv769091316:82 , srv931253118:82 , srv1256289722:83 , srv1256679487:83 , srv499443701:83 , srv744852200:83 , srv1331983984:84 , srv1655056478:84 , srv1682707636:84 , srv1713506116:84 , srv1723943055:84 , srv1866078466:84 , srv356368898:84 , srv367463610:84 , srv406810396:84 , srv421170616:84 , srv529392630:84 , srv654815810:84 , srv678967882:84 , srv685443373:84 , srv737937951:84 , srv740916019:84 , srv770741476:84 , srv1194367741:85 , srv1230353945:85 , srv1263822128:85 , srv1329273609:85 , srv1373792323:85 , srv1428687616:85 , srv1466055519:85 , srv1489700208:85 , srv1516033448:85 , srv1789648104:85 , srv1790983965:85 , srv1837715755:85 , srv2053845855:85 , srv2076504345:85 , srv2085583351:85 , srv366198930:85 , srv37707223:85 , srv396811548:85 , srv462936686:85 , srv467543906:85 , srv50161266:85 , srv538956242:85 , srv647636491:85 , srv809700605:85 , srv850555761:85 , srv916028113:85 , srv957242831:85 , srv1016444074:86 , srv1052614715:86 , srv1061754300:86 , srv1111740851:86 , srv1282422630:86 , srv130965354:86 , srv131309242:86 , srv1493021529:86 , srv1495866498:86 , srv1531822529:86 , srv1635535241:86 , srv1639557918:86 , srv1781043290:86 , srv1862500443:86 , srv1872608648:86 , srv1975022498:86 , srv2088278795:86 , srv305521630:86 , srv31464116:86 , srv318532246:86 , srv344801525:86 , srv348447034:86 , srv383607447:86 , srv444044030:86 , srv476086140:86 , srv698233454:86 , srv740284495:86 , srv894951630:86 , srv1142219635:87 , srv1148176017:87 , srv1299978469:87 , srv1621936389:87 , srv1682025021:87 , srv1730907443:87 , srv1867285004:87 , srv1893121957:87 , srv1954347294:87 , srv1995356768:87 , srv2054680576:87 , srv380425974:87 , srv490290401:87 , srv607506252:87 , srv613555182:87 , srv1411693917:88 , srv1560990594:88 , srv338800910:88 , srv453110845:88 , srv143527319:89 , srv649504141:546 }
2023-03-08 23:54:14,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:14,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:14,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:15,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:15,118 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006991702277396585); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011231669191044557); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006991702277396585); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,118 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-08 23:54:15,119 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001037025924784274, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006991702277396585); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011231669191044557); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006991702277396585); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-08 23:54:15,395 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 373 ms to try 5000 different iterations.  Found a solution that moves 274 regions; Going from a computed imbalance of 0.001037025924784274 to a new imbalance of 3.683355001599657E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.030444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,396 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1111740851:83 , srv1282422630:83 , srv1560990594:83 , srv1713506116:83 , srv1730907443:83 , srv344801525:83 , srv453110845:83 , srv1052614715:84 , srv1230353945:84 , srv1329273609:84 , srv143527319:84 , srv1531822529:84 , srv1621936389:84 , srv1635535241:84 , srv1682025021:84 , srv1781043290:84 , srv1862500443:84 , srv1872608648:84 , srv1893121957:84 , srv1954347294:84 , srv1995356768:84 , srv2053845855:84 , srv2054680576:84 , srv305521630:84 , srv31464116:84 , srv338800910:84 , srv367463610:84 , srv380425974:84 , srv421170616:84 , srv476086140:84 , srv50161266:84 , srv529392630:84 , srv538956242:84 , srv607506252:84 , srv613555182:84 , srv678967882:84 , srv698233454:84 , srv740284495:84 , srv740916019:84 , srv894951630:84 , srv931253118:84 , srv1016444074:85 , srv1061754300:85 , srv1142219635:85 , srv1256289722:85 , srv1256679487:85 , srv1263822128:85 , srv1299978469:85 , srv130965354:85 , srv1331983984:85 , srv1373792323:85 , srv1411693917:85 , srv1428687616:85 , srv1466055519:85 , srv1489700208:85 , srv1493021529:85 , srv1495866498:85 , srv1516033448:85 , srv1639557918:85 , srv1655056478:85 , srv1682707636:85 , srv1723943055:85 , srv1789648104:85 , srv1790983965:85 , srv1866078466:85 , srv1867285004:85 , srv1870030944:85 , srv1975022498:85 , srv2076504345:85 , srv2085583351:85 , srv318532246:85 , srv356368898:85 , srv366198930:85 , srv37707223:85 , srv383607447:85 , srv396811548:85 , srv406810396:85 , srv462936686:85 , srv467543906:85 , srv490290401:85 , srv499443701:85 , srv647636491:85 , srv685443373:85 , srv737937951:85 , srv744852200:85 , srv770741476:85 , srv850555761:85 , srv916028113:85 , srv957242831:85 , srv1148176017:86 , srv1194367741:86 , srv1837715755:86 , srv2088278795:86 , srv348447034:86 , srv444044030:86 , srv654815810:86 , srv769091316:86 , srv809700605:86 , srv131309242:87 , srv649504141:622 }
2023-03-08 23:54:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:15,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:15,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:15,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:15,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:15,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:15,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:15,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:15,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:15,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:15,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:15,529 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,529 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-08 23:54:15,530 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.491496100917148E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-08 23:54:15,757 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 324 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 3.491496100917148E-5 to a new imbalance of 8.490834924769608E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,758 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1061754300:83 , srv1194367741:83 , srv1256289722:83 , srv131309242:83 , srv1837715755:83 , srv2088278795:83 , srv348447034:83 , srv406810396:83 , srv444044030:83 , srv654815810:83 , srv769091316:83 , srv809700605:83 , srv1016444074:84 , srv1052614715:84 , srv1142219635:84 , srv1148176017:84 , srv1230353945:84 , srv1256679487:84 , srv1263822128:84 , srv1299978469:84 , srv130965354:84 , srv1331983984:84 , srv1373792323:84 , srv1411693917:84 , srv1428687616:84 , srv143527319:84 , srv1466055519:84 , srv1489700208:84 , srv1493021529:84 , srv1495866498:84 , srv1516033448:84 , srv1531822529:84 , srv1621936389:84 , srv1635535241:84 , srv1639557918:84 , srv1655056478:84 , srv1682025021:84 , srv1682707636:84 , srv1713506116:84 , srv1730907443:84 , srv1781043290:84 , srv1789648104:84 , srv1790983965:84 , srv1866078466:84 , srv1867285004:84 , srv1870030944:84 , srv1872608648:84 , srv1893121957:84 , srv1975022498:84 , srv1995356768:84 , srv2053845855:84 , srv2076504345:84 , srv2085583351:84 , srv305521630:84 , srv31464116:84 , srv318532246:84 , srv338800910:84 , srv356368898:84 , srv366198930:84 , srv37707223:84 , srv380425974:84 , srv383607447:84 , srv396811548:84 , srv421170616:84 , srv453110845:84 , srv462936686:84 , srv467543906:84 , srv476086140:84 , srv490290401:84 , srv499443701:84 , srv50161266:84 , srv538956242:84 , srv607506252:84 , srv613555182:84 , srv647636491:84 , srv678967882:84 , srv685443373:84 , srv698233454:84 , srv737937951:84 , srv740284495:84 , srv740916019:84 , srv770741476:84 , srv850555761:84 , srv894951630:84 , srv916028113:84 , srv931253118:84 , srv957242831:84 , srv1111740851:85 , srv1282422630:85 , srv1329273609:85 , srv1560990594:85 , srv1723943055:85 , srv1862500443:85 , srv1954347294:85 , srv2054680576:85 , srv344801525:85 , srv367463610:85 , srv529392630:85 , srv744852200:85 , srv649504141:684 }
2023-03-08 23:54:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:15,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1148176017=5, srv1682707636=33, srv1723943055=35, srv143527319=20, srv1781043290=37, srv338800910=58, srv529392630=78, srv1635535241=29, srv1142219635=4, srv1531822529=26, srv1489700208=22, srv31464116=56, srv1872608648=45, srv1329273609=15, srv344801525=59, srv1495866498=24, srv744852200=91, srv1639557918=30, srv1256289722=8, srv1428687616=19, srv1682025021=32, srv1411693917=18, srv366198930=62, srv737937951=88, srv380425974=65, srv538956242=79, srv1016444074=0, srv1466055519=21, srv2088278795=54, srv931253118=98, srv740916019=90, srv1373792323=17, srv1837715755=40, srv850555761=95, srv916028113=97, srv1560990594=27, srv654815810=84, srv770741476=93, srv476086140=74, srv1867285004=43, srv499443701=76, srv1230353945=7, srv131309242=14, srv1282422630=11, srv1713506116=34, srv2054680576=51, srv1493021529=23, srv678967882=85, srv1730907443=36, srv1870030944=44, srv2053845855=50, srv367463610=63, srv1263822128=10, srv305521630=55, srv318532246=57, srv1256679487=9, srv685443373=86, srv356368898=61, srv383607447=66, srv607506252=80, srv1790983965=39, srv1052614715=1, srv1995356768=49, srv649504141=83, srv453110845=71, srv613555182=81, srv1862500443=41, srv1655056478=31, srv396811548=67, srv462936686=72, srv1893121957=46, srv957242831=99, srv444044030=70, srv421170616=69, srv2076504345=52, srv647636491=82, srv809700605=94, srv1331983984=16, srv2085583351=53, srv1299978469=12, srv1954347294=47, srv1975022498=48, srv698233454=87, srv1061754300=2, srv1789648104=38, srv1866078466=42, srv130965354=13, srv740284495=89, srv467543906=73, srv50161266=77, srv1621936389=28, srv769091316=92, srv348447034=60, srv37707223=64, srv1194367741=6, srv1111740851=3, srv406810396=68, srv1516033448=25, srv894951630=96, srv490290401=75} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:15,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:15,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-08 23:54:15,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-08 23:54:15,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-08 23:54:15,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-08 23:54:15,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-08 23:54:15,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-08 23:54:15,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-08 23:54:15,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,887 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1061754300:83 , srv1194367741:83 , srv1256289722:83 , srv131309242:83 , srv1837715755:83 , srv2088278795:83 , srv348447034:83 , srv406810396:83 , srv444044030:83 , srv654815810:83 , srv769091316:83 , srv809700605:83 , srv1016444074:84 , srv1052614715:84 , srv1142219635:84 , srv1148176017:84 , srv1230353945:84 , srv1256679487:84 , srv1263822128:84 , srv1299978469:84 , srv130965354:84 , srv1331983984:84 , srv1373792323:84 , srv1411693917:84 , srv1428687616:84 , srv143527319:84 , srv1466055519:84 , srv1489700208:84 , srv1493021529:84 , srv1495866498:84 , srv1516033448:84 , srv1531822529:84 , srv1621936389:84 , srv1635535241:84 , srv1639557918:84 , srv1655056478:84 , srv1682025021:84 , srv1682707636:84 , srv1713506116:84 , srv1730907443:84 , srv1781043290:84 , srv1789648104:84 , srv1790983965:84 , srv1866078466:84 , srv1867285004:84 , srv1870030944:84 , srv1872608648:84 , srv1893121957:84 , srv1975022498:84 , srv1995356768:84 , srv2053845855:84 , srv2076504345:84 , srv2085583351:84 , srv305521630:84 , srv31464116:84 , srv318532246:84 , srv338800910:84 , srv356368898:84 , srv366198930:84 , srv37707223:84 , srv380425974:84 , srv383607447:84 , srv396811548:84 , srv421170616:84 , srv453110845:84 , srv462936686:84 , srv467543906:84 , srv476086140:84 , srv490290401:84 , srv499443701:84 , srv50161266:84 , srv538956242:84 , srv607506252:84 , srv613555182:84 , srv647636491:84 , srv678967882:84 , srv685443373:84 , srv698233454:84 , srv737937951:84 , srv740284495:84 , srv740916019:84 , srv770741476:84 , srv850555761:84 , srv894951630:84 , srv916028113:84 , srv931253118:84 , srv957242831:84 , srv1111740851:85 , srv1282422630:85 , srv1329273609:85 , srv1560990594:85 , srv1723943055:85 , srv1862500443:85 , srv1954347294:85 , srv2054680576:85 , srv344801525:85 , srv367463610:85 , srv529392630:85 , srv744852200:85 , srv649504141:684 }
2023-03-08 23:54:15,913 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 222) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=8163 (was 8663)
2023-03-08 23:54:15,921 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=170, AvailableMemoryMB=8162
2023-03-08 23:54:15,921 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-08 23:54:15,922 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-08 23:54:15,932 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001351608:84 , srv1033720317:84 , srv1100995569:84 , srv1206916750:84 , srv1218319935:84 , srv1235627045:84 , srv1269384190:84 , srv1280704192:84 , srv1337936359:84 , srv1383041578:84 , srv1390777476:84 , srv1462597915:84 , srv1842260428:84 , srv1844054991:84 , srv1896471676:84 , srv2107050575:84 , srv2147019065:84 , srv260471472:84 , srv395510263:84 , srv422725382:84 , srv467787863:84 , srv66467396:84 , srv71618439:84 , srv735039958:84 , srv739506888:84 , srv77187382:84 , srv825629468:84 , srv840149067:84 , srv908178875:84 , srv1023266918:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-08 23:54:15,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:54:15,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv735039958=24, srv1235627045=6, srv1218319935=5, srv1023266918=1, srv2107050575=16, srv1033720317=2, srv77187382=26, srv71618439=23, srv1390777476=11, srv1269384190=7, srv1383041578=10, srv1842260428=13, srv1844054991=14, srv422725382=20, srv825629468=27, srv908178875=29, srv739506888=25, srv66467396=22, srv1337936359=9, srv2147019065=17, srv840149067=28, srv1280704192=8, srv1896471676=15, srv1206916750=4, srv1100995569=3, srv260471472=18, srv467787863=21, srv395510263=19, srv1462597915=12, srv1001351608=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:54:15,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:54:15,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:54:15,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-08 23:54:15,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0908666240241813, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:54:15,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912479361029511, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0908666240241813, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-08 23:56:15,948 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6041645 different iterations.  Found a solution that moves 1838 regions; Going from a computed imbalance of 0.9912479361029511 to a new imbalance of 4.2899836916600065E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6807407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:56:15,954 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1001351608:90 , srv1023266918:90 , srv1033720317:90 , srv1100995569:90 , srv1206916750:90 , srv1218319935:90 , srv1235627045:90 , srv1269384190:90 , srv1280704192:90 , srv1337936359:90 , srv1383041578:90 , srv1390777476:90 , srv1462597915:90 , srv1842260428:90 , srv1844054991:90 , srv1896471676:90 , srv2107050575:90 , srv2147019065:90 , srv260471472:90 , srv395510263:90 , srv422725382:90 , srv467787863:90 , srv66467396:90 , srv71618439:90 , srv735039958:90 , srv739506888:90 , srv77187382:90 , srv825629468:90 , srv840149067:90 , srv908178875:90 }
2023-03-08 23:56:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-08 23:56:15,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv735039958=24, srv1235627045=6, srv1218319935=5, srv1023266918=1, srv2107050575=16, srv1033720317=2, srv77187382=26, srv71618439=23, srv1390777476=11, srv1269384190=7, srv1383041578=10, srv1842260428=13, srv1844054991=14, srv422725382=20, srv825629468=27, srv908178875=29, srv739506888=25, srv66467396=22, srv1337936359=9, srv2147019065=17, srv840149067=28, srv1280704192=8, srv1896471676=15, srv1206916750=4, srv1100995569=3, srv260471472=18, srv467787863=21, srv395510263=19, srv1462597915=12, srv1001351608=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-08 23:56:15,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-08 23:56:15,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-08 23:56:15,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-08 23:56:15,971 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-08 23:56:15,971 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1001351608:90 , srv1023266918:90 , srv1033720317:90 , srv1100995569:90 , srv1206916750:90 , srv1218319935:90 , srv1235627045:90 , srv1269384190:90 , srv1280704192:90 , srv1337936359:90 , srv1383041578:90 , srv1390777476:90 , srv1462597915:90 , srv1842260428:90 , srv1844054991:90 , srv1896471676:90 , srv2107050575:90 , srv2147019065:90 , srv260471472:90 , srv395510263:90 , srv422725382:90 , srv467787863:90 , srv66467396:90 , srv71618439:90 , srv735039958:90 , srv739506888:90 , srv77187382:90 , srv825629468:90 , srv840149067:90 , srv908178875:90 }
2023-03-08 23:56:15,984 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 227), ProcessCount=170 (was 170), AvailableMemoryMB=7863 (was 8162)