2023-03-14 03:54:19,976 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-14 03:54:20,286 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-14 03:54:20,389 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-14 03:54:20,445 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 03:54:20,452 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-14 03:54:20,465 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=169, AvailableMemoryMB=3185
2023-03-14 03:54:20,467 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 03:54:20,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-14 03:54:20,591 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1056681079:84 , srv1071659653:84 , srv1087491867:84 , srv1096645000:84 , srv1097397269:84 , srv1111594527:84 , srv1117940903:84 , srv112871379:84 , srv1133628845:84 , srv1193939576:84 , srv1226197301:84 , srv1233801148:84 , srv1260386732:84 , srv1263926484:84 , srv1264218635:84 , srv1293269981:84 , srv1299442507:84 , srv1303675818:84 , srv1308737835:84 , srv1319734549:84 , srv1355919989:84 , srv1444596791:84 , srv1480082735:84 , srv1506884787:84 , srv1519508314:84 , srv152716507:84 , srv1531765589:84 , srv1533029006:84 , srv160807615:84 , srv161438144:84 , srv1624670096:84 , srv16728991:84 , srv1681773334:84 , srv1684540715:84 , srv1689808977:84 , srv1689951113:84 , srv1701500867:84 , srv1710953276:84 , srv1723288852:84 , srv181900875:84 , srv1821370283:84 , srv1827809542:84 , srv1861234877:84 , srv1867044753:84 , srv1947608980:84 , srv1958450986:84 , srv1966821707:84 , srv1980082062:84 , srv198565278:84 , srv201085143:84 , srv2012201809:84 , srv2032275979:84 , srv2032493536:84 , srv2054628209:84 , srv2094559076:84 , srv2107669424:84 , srv2134807217:84 , srv223101577:84 , srv228611065:84 , srv238144207:84 , srv263991668:84 , srv273820529:84 , srv313852787:84 , srv337765257:84 , srv344526144:84 , srv354898444:84 , srv355687616:84 , srv381685827:84 , srv402775364:84 , srv405216124:84 , srv43169885:84 , srv463513013:84 , srv466717313:84 , srv47877708:84 , srv543356630:84 , srv565258913:84 , srv584625370:84 , srv623054757:84 , srv647724073:84 , srv680434314:84 , srv681099444:84 , srv683152320:84 , srv697774725:84 , srv710351423:84 , srv723628443:84 , srv737381083:84 , srv746766667:84 , srv775121181:84 , srv799941303:84 , srv824849232:84 , srv832747466:84 , srv842480364:84 , srv851324653:84 , srv865332829:84 , srv866615867:84 , srv899913564:84 , srv901113902:84 , srv908802635:84 , srv920622596:84 , srv1847867268:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-14 03:54:20,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:20,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv161438144=29, srv152716507=25, srv1533029006=27, srv1264218635=14, srv680434314=80, srv901113902=97, srv1303675818=17, srv2032275979=52, srv463513013=72, srv1319734549=19, srv899913564=96, srv1480082735=22, srv1701500867=36, srv228611065=59, srv1681773334=32, srv1097397269=4, srv160807615=28, srv2107669424=56, srv697774725=83, srv112871379=7, srv354898444=66, srv2012201809=51, srv1308737835=18, srv223101577=58, srv1117940903=6, srv405216124=70, srv623054757=78, srv181900875=39, srv851324653=93, srv1847867268=42, srv201085143=50, srv1531765589=26, srv1689951113=35, srv799941303=89, srv908802635=98, srv1096645000=3, srv1827809542=41, srv1355919989=20, srv1263926484=13, srv584625370=77, srv1299442507=16, srv1710953276=37, srv263991668=61, srv565258913=76, srv746766667=87, srv43169885=71, srv1293269981=15, srv737381083=86, srv273820529=62, srv355687616=67, srv47877708=74, srv1506884787=23, srv865332829=94, srv1689808977=34, srv710351423=84, srv1233801148=11, srv2032493536=53, srv1071659653=1, srv775121181=88, srv543356630=75, srv2134807217=57, srv683152320=82, srv1444596791=21, srv1226197301=10, srv313852787=63, srv1624670096=30, srv1821370283=40, srv1684540715=33, srv466717313=73, srv824849232=90, srv1133628845=8, srv1087491867=2, srv842480364=92, srv866615867=95, srv1193939576=9, srv337765257=64, srv1723288852=38, srv1111594527=5, srv238144207=60, srv647724073=79, srv198565278=49, srv2094559076=55, srv16728991=31, srv402775364=69, srv1861234877=43, srv1947608980=45, srv723628443=85, srv681099444=81, srv344526144=65, srv1519508314=24, srv832747466=91, srv1966821707=47, srv2054628209=54, srv920622596=99, srv1958450986=46, srv1980082062=48, srv381685827=68, srv1867044753=44, srv1260386732=12, srv1056681079=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 03:54:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 03:54:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 03:54:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 03:54:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 03:54:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 03:54:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 03:54:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 03:54:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 03:54:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 03:54:20,983 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:20,983 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 03:54:20,984 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 03:54:21,641 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 931 ms to try 5000 different iterations.  Found a solution that moves 4682 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17643148779222897. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037275327713069434, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056126360564073184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5202222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037275327713069434, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17175, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23708333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:21,664 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1071659653:81 , srv1133628845:82 , srv337765257:82 , srv824849232:82 , srv1111594527:83 , srv1264218635:83 , srv1519508314:83 , srv680434314:83 , srv697774725:83 , srv842480364:83 , srv1096645000:84 , srv1097397269:84 , srv1263926484:84 , srv1319734549:84 , srv1689808977:84 , srv1966821707:84 , srv198565278:84 , srv228611065:84 , srv565258913:84 , srv623054757:84 , srv112871379:85 , srv1293269981:85 , srv1444596791:85 , srv1533029006:85 , srv160807615:85 , srv161438144:85 , srv1710953276:85 , srv1947608980:85 , srv1980082062:85 , srv2032275979:85 , srv2054628209:85 , srv2134807217:85 , srv273820529:85 , srv313852787:85 , srv381685827:85 , srv405216124:85 , srv543356630:85 , srv584625370:85 , srv681099444:85 , srv908802635:85 , srv1117940903:86 , srv1193939576:86 , srv1480082735:86 , srv181900875:86 , srv1821370283:86 , srv1861234877:86 , srv2032493536:86 , srv344526144:86 , srv43169885:86 , srv683152320:86 , srv723628443:86 , srv1087491867:87 , srv1260386732:87 , srv1299442507:87 , srv1303675818:87 , srv1624670096:87 , srv1681773334:87 , srv2012201809:87 , srv2094559076:87 , srv355687616:87 , srv402775364:87 , srv466717313:87 , srv832747466:87 , srv865332829:87 , srv866615867:87 , srv920622596:87 , srv1056681079:88 , srv1233801148:88 , srv1355919989:88 , srv1684540715:88 , srv1701500867:88 , srv1867044753:88 , srv1958450986:88 , srv263991668:88 , srv647724073:88 , srv899913564:88 , srv1226197301:89 , srv1506884787:89 , srv223101577:89 , srv354898444:89 , srv710351423:89 , srv746766667:89 , srv775121181:89 , srv799941303:89 , srv851324653:89 , srv16728991:90 , srv201085143:90 , srv901113902:90 , srv1689951113:91 , srv1723288852:91 , srv1827809542:91 , srv2107669424:91 , srv238144207:91 , srv463513013:91 , srv47877708:91 , srv737381083:92 , srv1308737835:93 , srv152716507:94 , srv1531765589:96 , srv1847867268:421 }
2023-03-14 03:54:21,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:21,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv161438144=29, srv152716507=25, srv1533029006=27, srv1264218635=14, srv680434314=80, srv901113902=97, srv1303675818=17, srv2032275979=52, srv463513013=72, srv1319734549=19, srv899913564=96, srv1480082735=22, srv1701500867=36, srv228611065=59, srv1681773334=32, srv1097397269=4, srv160807615=28, srv2107669424=56, srv697774725=83, srv112871379=7, srv354898444=66, srv2012201809=51, srv1308737835=18, srv223101577=58, srv1117940903=6, srv405216124=70, srv623054757=78, srv181900875=39, srv851324653=93, srv1847867268=42, srv201085143=50, srv1531765589=26, srv1689951113=35, srv799941303=89, srv908802635=98, srv1096645000=3, srv1827809542=41, srv1355919989=20, srv1263926484=13, srv584625370=77, srv1299442507=16, srv1710953276=37, srv263991668=61, srv565258913=76, srv746766667=87, srv43169885=71, srv1293269981=15, srv737381083=86, srv273820529=62, srv355687616=67, srv47877708=74, srv1506884787=23, srv865332829=94, srv1689808977=34, srv710351423=84, srv1233801148=11, srv2032493536=53, srv1071659653=1, srv775121181=88, srv543356630=75, srv2134807217=57, srv683152320=82, srv1444596791=21, srv1226197301=10, srv313852787=63, srv1624670096=30, srv1821370283=40, srv1684540715=33, srv466717313=73, srv824849232=90, srv1133628845=8, srv1087491867=2, srv842480364=92, srv866615867=95, srv1193939576=9, srv337765257=64, srv1723288852=38, srv1111594527=5, srv238144207=60, srv647724073=79, srv198565278=49, srv2094559076=55, srv16728991=31, srv402775364=69, srv1861234877=43, srv1947608980=45, srv723628443=85, srv681099444=81, srv344526144=65, srv1519508314=24, srv832747466=91, srv1966821707=47, srv2054628209=54, srv920622596=99, srv1958450986=46, srv1980082062=48, srv381685827=68, srv1867044753=44, srv1260386732=12, srv1056681079=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:21,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:21,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:21,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:21,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:21,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:21,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:21,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:21,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 03:54:21,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 03:54:21,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 03:54:21,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 03:54:21,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 03:54:21,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 03:54:21,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 03:54:21,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 03:54:21,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 03:54:21,860 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037275327713069434, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056126360564073184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037275327713069434, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17175, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23708333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:21,860 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 03:54:21,860 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1763987037275211, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037275327713069434, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056126360564073184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037275327713069434, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17175, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23708333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 03:54:22,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 432 ms to try 5000 different iterations.  Found a solution that moves 2645 regions; Going from a computed imbalance of 0.1763987037275211 to a new imbalance of 0.01372503640425709. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018700786517846863); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03494745133278946, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29388888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018700786517846863); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:22,174 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv152716507:75 , srv1308737835:76 , srv1531765589:78 , srv238144207:80 , srv737381083:80 , srv201085143:81 , srv2107669424:81 , srv463513013:81 , srv47877708:81 , srv775121181:81 , srv1506884787:82 , srv1689951113:82 , srv1827809542:82 , srv1684540715:83 , srv1701500867:83 , srv263991668:83 , srv799941303:83 , srv866615867:83 , srv899913564:83 , srv1233801148:84 , srv1723288852:84 , srv2012201809:84 , srv344526144:84 , srv710351423:84 , srv746766667:84 , srv901113902:84 , srv1056681079:85 , srv1193939576:85 , srv1226197301:85 , srv1263926484:85 , srv1299442507:85 , srv1355919989:85 , srv16728991:85 , srv1867044753:85 , srv1958450986:85 , srv223101577:85 , srv313852787:85 , srv354898444:85 , srv355687616:85 , srv402775364:85 , srv584625370:85 , srv647724073:85 , srv851324653:85 , srv865332829:85 , srv1087491867:86 , srv1681773334:86 , srv1861234877:86 , srv1947608980:86 , srv2032275979:86 , srv2094559076:86 , srv273820529:86 , srv466717313:86 , srv543356630:86 , srv623054757:86 , srv832747466:86 , srv908802635:86 , srv1260386732:87 , srv160807615:87 , srv181900875:87 , srv1821370283:87 , srv1980082062:87 , srv2054628209:87 , srv2134807217:87 , srv228611065:87 , srv381685827:87 , srv43169885:87 , srv683152320:87 , srv723628443:87 , srv920622596:87 , srv1096645000:88 , srv1111594527:88 , srv1117940903:88 , srv112871379:88 , srv1133628845:88 , srv1303675818:88 , srv1444596791:88 , srv1533029006:88 , srv161438144:88 , srv1624670096:88 , srv1710953276:88 , srv1966821707:88 , srv198565278:88 , srv2032493536:88 , srv697774725:88 , srv824849232:88 , srv842480364:88 , srv1097397269:89 , srv1293269981:89 , srv1319734549:89 , srv1480082735:89 , srv1689808977:89 , srv405216124:89 , srv565258913:89 , srv1071659653:90 , srv337765257:90 , srv681099444:90 , srv1264218635:91 , srv1519508314:91 , srv680434314:91 , srv1847867268:519 }
2023-03-14 03:54:22,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:22,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv161438144=29, srv152716507=25, srv1533029006=27, srv1264218635=14, srv680434314=80, srv901113902=97, srv1303675818=17, srv2032275979=52, srv463513013=72, srv1319734549=19, srv899913564=96, srv1480082735=22, srv1701500867=36, srv228611065=59, srv1681773334=32, srv1097397269=4, srv160807615=28, srv2107669424=56, srv697774725=83, srv112871379=7, srv354898444=66, srv2012201809=51, srv1308737835=18, srv223101577=58, srv1117940903=6, srv405216124=70, srv623054757=78, srv181900875=39, srv851324653=93, srv1847867268=42, srv201085143=50, srv1531765589=26, srv1689951113=35, srv799941303=89, srv908802635=98, srv1096645000=3, srv1827809542=41, srv1355919989=20, srv1263926484=13, srv584625370=77, srv1299442507=16, srv1710953276=37, srv263991668=61, srv565258913=76, srv746766667=87, srv43169885=71, srv1293269981=15, srv737381083=86, srv273820529=62, srv355687616=67, srv47877708=74, srv1506884787=23, srv865332829=94, srv1689808977=34, srv710351423=84, srv1233801148=11, srv2032493536=53, srv1071659653=1, srv775121181=88, srv543356630=75, srv2134807217=57, srv683152320=82, srv1444596791=21, srv1226197301=10, srv313852787=63, srv1624670096=30, srv1821370283=40, srv1684540715=33, srv466717313=73, srv824849232=90, srv1133628845=8, srv1087491867=2, srv842480364=92, srv866615867=95, srv1193939576=9, srv337765257=64, srv1723288852=38, srv1111594527=5, srv238144207=60, srv647724073=79, srv198565278=49, srv2094559076=55, srv16728991=31, srv402775364=69, srv1861234877=43, srv1947608980=45, srv723628443=85, srv681099444=81, srv344526144=65, srv1519508314=24, srv832747466=91, srv1966821707=47, srv2054628209=54, srv920622596=99, srv1958450986=46, srv1980082062=48, srv381685827=68, srv1867044753=44, srv1260386732=12, srv1056681079=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:22,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:22,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:22,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 03:54:22,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 03:54:22,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 03:54:22,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 03:54:22,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 03:54:22,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 03:54:22,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 03:54:22,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 03:54:22,316 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018700786517846863); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03494745133278946, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018700786517846863); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:22,317 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 03:54:22,317 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013706515718406535, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.018700786517846863); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03494745133278946, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.018700786517846863); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 03:54:22,592 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 378 ms to try 5000 different iterations.  Found a solution that moves 1022 regions; Going from a computed imbalance of 0.013706515718406535 to a new imbalance of 6.185552417529645E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004771888417305602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.11355555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004771888417305602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:22,595 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1480082735:82 , srv1624670096:82 , srv16728991:82 , srv1226197301:83 , srv1260386732:83 , srv1293269981:83 , srv1303675818:83 , srv1531765589:83 , srv1723288852:83 , srv2032493536:83 , srv223101577:83 , srv354898444:83 , srv405216124:83 , srv851324653:83 , srv1087491867:84 , srv1097397269:84 , srv1117940903:84 , srv1264218635:84 , srv1444596791:84 , srv161438144:84 , srv1681773334:84 , srv1689808977:84 , srv1710953276:84 , srv181900875:84 , srv565258913:84 , srv647724073:84 , srv680434314:84 , srv681099444:84 , srv710351423:84 , srv746766667:84 , srv832747466:84 , srv901113902:84 , srv920622596:84 , srv1056681079:85 , srv1096645000:85 , srv112871379:85 , srv1519508314:85 , srv1533029006:85 , srv160807615:85 , srv1689951113:85 , srv1821370283:85 , srv1827809542:85 , srv1867044753:85 , srv1958450986:85 , srv1966821707:85 , srv1980082062:85 , srv198565278:85 , srv2012201809:85 , srv2054628209:85 , srv355687616:85 , srv402775364:85 , srv43169885:85 , srv463513013:85 , srv466717313:85 , srv47877708:85 , srv683152320:85 , srv737381083:85 , srv1111594527:86 , srv1193939576:86 , srv1233801148:86 , srv1299442507:86 , srv1319734549:86 , srv1355919989:86 , srv1506884787:86 , srv1861234877:86 , srv2032275979:86 , srv2107669424:86 , srv2134807217:86 , srv337765257:86 , srv381685827:86 , srv543356630:86 , srv723628443:86 , srv799941303:86 , srv842480364:86 , srv865332829:86 , srv899913564:86 , srv908802635:86 , srv1071659653:87 , srv1684540715:87 , srv1701500867:87 , srv1947608980:87 , srv201085143:87 , srv2094559076:87 , srv228611065:87 , srv238144207:87 , srv263991668:87 , srv273820529:87 , srv313852787:87 , srv344526144:87 , srv697774725:87 , srv866615867:87 , srv1133628845:88 , srv152716507:88 , srv584625370:88 , srv623054757:88 , srv824849232:88 , srv1263926484:89 , srv1308737835:89 , srv775121181:89 , srv1847867268:560 }
2023-03-14 03:54:22,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:22,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv161438144=29, srv152716507=25, srv1533029006=27, srv1264218635=14, srv680434314=80, srv901113902=97, srv1303675818=17, srv2032275979=52, srv463513013=72, srv1319734549=19, srv899913564=96, srv1480082735=22, srv1701500867=36, srv228611065=59, srv1681773334=32, srv1097397269=4, srv160807615=28, srv2107669424=56, srv697774725=83, srv112871379=7, srv354898444=66, srv2012201809=51, srv1308737835=18, srv223101577=58, srv1117940903=6, srv405216124=70, srv623054757=78, srv181900875=39, srv851324653=93, srv1847867268=42, srv201085143=50, srv1531765589=26, srv1689951113=35, srv799941303=89, srv908802635=98, srv1096645000=3, srv1827809542=41, srv1355919989=20, srv1263926484=13, srv584625370=77, srv1299442507=16, srv1710953276=37, srv263991668=61, srv565258913=76, srv746766667=87, srv43169885=71, srv1293269981=15, srv737381083=86, srv273820529=62, srv355687616=67, srv47877708=74, srv1506884787=23, srv865332829=94, srv1689808977=34, srv710351423=84, srv1233801148=11, srv2032493536=53, srv1071659653=1, srv775121181=88, srv543356630=75, srv2134807217=57, srv683152320=82, srv1444596791=21, srv1226197301=10, srv313852787=63, srv1624670096=30, srv1821370283=40, srv1684540715=33, srv466717313=73, srv824849232=90, srv1133628845=8, srv1087491867=2, srv842480364=92, srv866615867=95, srv1193939576=9, srv337765257=64, srv1723288852=38, srv1111594527=5, srv238144207=60, srv647724073=79, srv198565278=49, srv2094559076=55, srv16728991=31, srv402775364=69, srv1861234877=43, srv1947608980=45, srv723628443=85, srv681099444=81, srv344526144=65, srv1519508314=24, srv832747466=91, srv1966821707=47, srv2054628209=54, srv920622596=99, srv1958450986=46, srv1980082062=48, srv381685827=68, srv1867044753=44, srv1260386732=12, srv1056681079=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:22,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:22,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 03:54:22,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 03:54:22,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 03:54:22,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 03:54:22,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 03:54:22,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 03:54:22,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 03:54:22,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 03:54:22,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004771888417305602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004771888417305602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:22,743 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-14 03:54:22,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.113990448005E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004771888417305602); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0073854894587599635); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004771888417305602); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-14 03:54:23,058 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 410 ms to try 5000 different iterations.  Found a solution that moves 265 regions; Going from a computed imbalance of 6.113990448005E-4 to a new imbalance of 2.1059315158718475E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.029444444444444443, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:23,059 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2094559076:82 , srv112871379:83 , srv1308737835:83 , srv1319734549:83 , srv1355919989:83 , srv1861234877:83 , srv201085143:83 , srv273820529:83 , srv584625370:83 , srv681099444:83 , srv683152320:83 , srv723628443:83 , srv775121181:83 , srv1056681079:84 , srv1071659653:84 , srv1117940903:84 , srv1133628845:84 , srv1233801148:84 , srv1263926484:84 , srv1264218635:84 , srv1299442507:84 , srv1303675818:84 , srv1519508314:84 , srv1533029006:84 , srv1684540715:84 , srv1689951113:84 , srv1723288852:84 , srv1827809542:84 , srv1867044753:84 , srv1947608980:84 , srv1958450986:84 , srv2134807217:84 , srv228611065:84 , srv263991668:84 , srv337765257:84 , srv354898444:84 , srv381685827:84 , srv43169885:84 , srv466717313:84 , srv623054757:84 , srv697774725:84 , srv799941303:84 , srv824849232:84 , srv865332829:84 , srv901113902:84 , srv908802635:84 , srv920622596:84 , srv1087491867:85 , srv1097397269:85 , srv1111594527:85 , srv1193939576:85 , srv1226197301:85 , srv1260386732:85 , srv1293269981:85 , srv1480082735:85 , srv1506884787:85 , srv152716507:85 , srv1531765589:85 , srv161438144:85 , srv1624670096:85 , srv16728991:85 , srv1681773334:85 , srv1701500867:85 , srv1710953276:85 , srv181900875:85 , srv1821370283:85 , srv1966821707:85 , srv1980082062:85 , srv198565278:85 , srv2012201809:85 , srv2032275979:85 , srv2032493536:85 , srv2054628209:85 , srv2107669424:85 , srv223101577:85 , srv238144207:85 , srv313852787:85 , srv344526144:85 , srv355687616:85 , srv402775364:85 , srv405216124:85 , srv463513013:85 , srv47877708:85 , srv543356630:85 , srv565258913:85 , srv647724073:85 , srv680434314:85 , srv710351423:85 , srv737381083:85 , srv746766667:85 , srv832747466:85 , srv842480364:85 , srv851324653:85 , srv866615867:85 , srv899913564:85 , srv1096645000:86 , srv1444596791:86 , srv160807615:86 , srv1689808977:86 , srv1847867268:642 }
2023-03-14 03:54:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv161438144=29, srv152716507=25, srv1533029006=27, srv1264218635=14, srv680434314=80, srv901113902=97, srv1303675818=17, srv2032275979=52, srv463513013=72, srv1319734549=19, srv899913564=96, srv1480082735=22, srv1701500867=36, srv228611065=59, srv1681773334=32, srv1097397269=4, srv160807615=28, srv2107669424=56, srv697774725=83, srv112871379=7, srv354898444=66, srv2012201809=51, srv1308737835=18, srv223101577=58, srv1117940903=6, srv405216124=70, srv623054757=78, srv181900875=39, srv851324653=93, srv1847867268=42, srv201085143=50, srv1531765589=26, srv1689951113=35, srv799941303=89, srv908802635=98, srv1096645000=3, srv1827809542=41, srv1355919989=20, srv1263926484=13, srv584625370=77, srv1299442507=16, srv1710953276=37, srv263991668=61, srv565258913=76, srv746766667=87, srv43169885=71, srv1293269981=15, srv737381083=86, srv273820529=62, srv355687616=67, srv47877708=74, srv1506884787=23, srv865332829=94, srv1689808977=34, srv710351423=84, srv1233801148=11, srv2032493536=53, srv1071659653=1, srv775121181=88, srv543356630=75, srv2134807217=57, srv683152320=82, srv1444596791=21, srv1226197301=10, srv313852787=63, srv1624670096=30, srv1821370283=40, srv1684540715=33, srv466717313=73, srv824849232=90, srv1133628845=8, srv1087491867=2, srv842480364=92, srv866615867=95, srv1193939576=9, srv337765257=64, srv1723288852=38, srv1111594527=5, srv238144207=60, srv647724073=79, srv198565278=49, srv2094559076=55, srv16728991=31, srv402775364=69, srv1861234877=43, srv1947608980=45, srv723628443=85, srv681099444=81, srv344526144=65, srv1519508314=24, srv832747466=91, srv1966821707=47, srv2054628209=54, srv920622596=99, srv1958450986=46, srv1980082062=48, srv381685827=68, srv1867044753=44, srv1260386732=12, srv1056681079=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:23,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:23,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-14 03:54:23,085 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-14 03:54:23,086 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-14 03:54:23,087 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-14 03:54:23,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-14 03:54:23,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-14 03:54:23,088 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-14 03:54:23,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-14 03:54:23,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-14 03:54:23,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-14 03:54:23,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-14 03:54:23,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-14 03:54:23,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-14 03:54:23,217 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.920374549883289E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:23,218 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2094559076:82 , srv112871379:83 , srv1308737835:83 , srv1319734549:83 , srv1355919989:83 , srv1861234877:83 , srv201085143:83 , srv273820529:83 , srv584625370:83 , srv681099444:83 , srv683152320:83 , srv723628443:83 , srv775121181:83 , srv1056681079:84 , srv1071659653:84 , srv1117940903:84 , srv1133628845:84 , srv1233801148:84 , srv1263926484:84 , srv1264218635:84 , srv1299442507:84 , srv1303675818:84 , srv1519508314:84 , srv1533029006:84 , srv1684540715:84 , srv1689951113:84 , srv1723288852:84 , srv1827809542:84 , srv1867044753:84 , srv1947608980:84 , srv1958450986:84 , srv2134807217:84 , srv228611065:84 , srv263991668:84 , srv337765257:84 , srv354898444:84 , srv381685827:84 , srv43169885:84 , srv466717313:84 , srv623054757:84 , srv697774725:84 , srv799941303:84 , srv824849232:84 , srv865332829:84 , srv901113902:84 , srv908802635:84 , srv920622596:84 , srv1087491867:85 , srv1097397269:85 , srv1111594527:85 , srv1193939576:85 , srv1226197301:85 , srv1260386732:85 , srv1293269981:85 , srv1480082735:85 , srv1506884787:85 , srv152716507:85 , srv1531765589:85 , srv161438144:85 , srv1624670096:85 , srv16728991:85 , srv1681773334:85 , srv1701500867:85 , srv1710953276:85 , srv181900875:85 , srv1821370283:85 , srv1966821707:85 , srv1980082062:85 , srv198565278:85 , srv2012201809:85 , srv2032275979:85 , srv2032493536:85 , srv2054628209:85 , srv2107669424:85 , srv223101577:85 , srv238144207:85 , srv313852787:85 , srv344526144:85 , srv355687616:85 , srv402775364:85 , srv405216124:85 , srv463513013:85 , srv47877708:85 , srv543356630:85 , srv565258913:85 , srv647724073:85 , srv680434314:85 , srv710351423:85 , srv737381083:85 , srv746766667:85 , srv832747466:85 , srv842480364:85 , srv851324653:85 , srv866615867:85 , srv899913564:85 , srv1096645000:86 , srv1444596791:86 , srv160807615:86 , srv1689808977:86 , srv1847867268:642 }
2023-03-14 03:54:23,257 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=332 (was 308) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=2691 (was 3185)
2023-03-14 03:54:23,268 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=332, ProcessCount=169, AvailableMemoryMB=2690
2023-03-14 03:54:23,269 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-14 03:54:23,269 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-14 03:54:23,279 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1092415853:84 , srv1095418146:84 , srv1114383750:84 , srv1268541702:84 , srv1325617196:84 , srv135563725:84 , srv1487811183:84 , srv1645226920:84 , srv1754057739:84 , srv1860041937:84 , srv1962125319:84 , srv2030116204:84 , srv2039394778:84 , srv2046672133:84 , srv211295056:84 , srv2138340213:84 , srv218046245:84 , srv254183079:84 , srv457521930:84 , srv459847026:84 , srv468000501:84 , srv510582511:84 , srv622740234:84 , srv78618773:84 , srv817375314:84 , srv826362518:84 , srv86045570:84 , srv912131881:84 , srv915184690:84 , srv1960668572:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-14 03:54:23,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:54:23,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv468000501=21, srv817375314=25, srv1114383750=2, srv1487811183=6, srv78618773=24, srv1860041937=9, srv1092415853=0, srv915184690=29, srv2030116204=12, srv510582511=22, srv2039394778=13, srv912131881=28, srv459847026=20, srv1962125319=11, srv86045570=27, srv1960668572=10, srv457521930=19, srv135563725=5, srv2138340213=16, srv1325617196=4, srv211295056=15, srv826362518=26, srv254183079=18, srv1645226920=7, srv1268541702=3, srv1095418146=1, srv622740234=23, srv218046245=17, srv1754057739=8, srv2046672133=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:54:23,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:54:23,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-14 03:54:23,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.095842701176398, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:54:23,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912495040486135, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.095842701176398, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-14 03:56:23,295 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6757208 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912495040486135 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:56:23,301 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1092415853:90 , srv1095418146:90 , srv1114383750:90 , srv1268541702:90 , srv1325617196:90 , srv135563725:90 , srv1487811183:90 , srv1645226920:90 , srv1754057739:90 , srv1860041937:90 , srv1960668572:90 , srv1962125319:90 , srv2030116204:90 , srv2039394778:90 , srv2046672133:90 , srv211295056:90 , srv2138340213:90 , srv218046245:90 , srv254183079:90 , srv457521930:90 , srv459847026:90 , srv468000501:90 , srv510582511:90 , srv622740234:90 , srv78618773:90 , srv817375314:90 , srv826362518:90 , srv86045570:90 , srv912131881:90 , srv915184690:90 }
2023-03-14 03:56:23,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-14 03:56:23,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv468000501=21, srv817375314=25, srv1114383750=2, srv1487811183=6, srv78618773=24, srv1860041937=9, srv1092415853=0, srv915184690=29, srv2030116204=12, srv510582511=22, srv2039394778=13, srv912131881=28, srv459847026=20, srv1962125319=11, srv86045570=27, srv1960668572=10, srv457521930=19, srv135563725=5, srv2138340213=16, srv1325617196=4, srv211295056=15, srv826362518=26, srv254183079=18, srv1645226920=7, srv1268541702=3, srv1095418146=1, srv622740234=23, srv218046245=17, srv1754057739=8, srv2046672133=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-14 03:56:23,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-14 03:56:23,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-14 03:56:23,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-14 03:56:23,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-14 03:56:23,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-14 03:56:23,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-14 03:56:23,318 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1092415853:90 , srv1095418146:90 , srv1114383750:90 , srv1268541702:90 , srv1325617196:90 , srv135563725:90 , srv1487811183:90 , srv1645226920:90 , srv1754057739:90 , srv1860041937:90 , srv1960668572:90 , srv1962125319:90 , srv2030116204:90 , srv2039394778:90 , srv2046672133:90 , srv211295056:90 , srv2138340213:90 , srv218046245:90 , srv254183079:90 , srv457521930:90 , srv459847026:90 , srv468000501:90 , srv510582511:90 , srv622740234:90 , srv78618773:90 , srv817375314:90 , srv826362518:90 , srv86045570:90 , srv912131881:90 , srv915184690:90 }
2023-03-14 03:56:23,330 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=142 (was 332), ProcessCount=169 (was 169), AvailableMemoryMB=2398 (was 2690)