2023-05-16 16:53:25,714 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-16 16:53:26,030 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-16 16:53:26,148 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-16 16:53:26,211 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 16:53:26,219 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-16 16:53:26,235 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=415, ProcessCount=176, AvailableMemoryMB=11897
2023-05-16 16:53:26,237 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 16:53:26,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-16 16:53:26,367 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011088674:84 , srv1020399437:84 , srv1033891417:84 , srv1040652319:84 , srv1063921129:84 , srv1067512075:84 , srv1079478287:84 , srv1090164700:84 , srv114612743:84 , srv1147716714:84 , srv1161694338:84 , srv1164849861:84 , srv12049046:84 , srv124620357:84 , srv1246738502:84 , srv1249869182:84 , srv1262752896:84 , srv1342375300:84 , srv1359379584:84 , srv1366339462:84 , srv1379689369:84 , srv1417857483:84 , srv1479935651:84 , srv1493615921:84 , srv1520785322:84 , srv1564930403:84 , srv1597135585:84 , srv1599668646:84 , srv1610159946:84 , srv1640841558:84 , srv1653854934:84 , srv166389512:84 , srv1675559717:84 , srv1684526744:84 , srv1689966932:84 , srv1693713624:84 , srv1707092085:84 , srv1757864304:84 , srv1768165078:84 , srv1777884879:84 , srv1780796090:84 , srv1785675713:84 , srv1792874021:84 , srv1803792158:84 , srv1890869832:84 , srv1903233800:84 , srv1907734422:84 , srv1931865994:84 , srv1938840447:84 , srv1955482195:84 , srv2032344461:84 , srv203415186:84 , srv204808782:84 , srv204841728:84 , srv2067861830:84 , srv2079126881:84 , srv2100672922:84 , srv215586354:84 , srv223485644:84 , srv229499859:84 , srv2454807:84 , srv252611734:84 , srv263034954:84 , srv307848856:84 , srv320922299:84 , srv336682823:84 , srv363849855:84 , srv370362442:84 , srv4055176:84 , srv446881812:84 , srv471985223:84 , srv472724889:84 , srv503186585:84 , srv529643373:84 , srv600058728:84 , srv639860541:84 , srv672446821:84 , srv706900250:84 , srv725618115:84 , srv72578049:84 , srv73052772:84 , srv763371005:84 , srv763784796:84 , srv790854183:84 , srv791717211:84 , srv815655821:84 , srv824152852:84 , srv827204954:84 , srv841148049:84 , srv893774507:84 , srv898631494:84 , srv911252193:84 , srv918457790:84 , srv932907916:84 , srv968046652:84 , srv985126142:84 , srv998354672:84 , srv998437746:84 , srv999139865:84 , srv1940280756:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-16 16:53:26,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:26,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1599668646=27, srv1597135585=26, srv2100672922=57, srv998354672=97, srv1033891417=2, srv503186585=73, srv203415186=52, srv363849855=67, srv1520785322=24, srv600058728=75, srv2079126881=56, srv1940280756=49, srv1777884879=39, srv2067861830=55, srv1792874021=42, srv529643373=74, srv73052772=81, srv1780796090=40, srv166389512=31, srv841148049=89, srv1689966932=34, srv1246738502=14, srv999139865=99, srv124620357=13, srv1693713624=35, srv827204954=88, srv1342375300=17, srv229499859=60, srv215586354=58, srv672446821=77, srv791717211=85, srv1161694338=10, srv1684526744=33, srv1785675713=41, srv763371005=82, srv1011088674=0, srv72578049=80, srv1493615921=23, srv204841728=54, srv252611734=62, srv1249869182=15, srv1359379584=18, srv985126142=96, srv1417857483=21, srv1040652319=3, srv1653854934=30, srv263034954=63, srv204808782=53, srv1147716714=9, srv1675559717=32, srv998437746=98, srv1366339462=19, srv1020399437=1, srv893774507=90, srv1757864304=37, srv370362442=68, srv1707092085=36, srv223485644=59, srv2454807=61, srv706900250=78, srv911252193=92, srv1890869832=44, srv4055176=69, srv1768165078=38, srv320922299=65, srv1803792158=43, srv1564930403=25, srv968046652=95, srv898631494=91, srv824152852=87, srv790854183=84, srv1955482195=50, srv1931865994=47, srv815655821=86, srv1610159946=28, srv763784796=83, srv446881812=70, srv932907916=94, srv1640841558=29, srv1379689369=20, srv307848856=64, srv1090164700=7, srv336682823=66, srv12049046=12, srv114612743=8, srv1907734422=46, srv472724889=72, srv639860541=76, srv1063921129=4, srv2032344461=51, srv1938840447=48, srv725618115=79, srv1079478287=6, srv918457790=93, srv1903233800=45, srv1479935651=22, srv1262752896=16, srv1067512075=5, srv471985223=71, srv1164849861=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:26,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:26,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:26,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 16:53:26,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 16:53:26,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 16:53:26,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 16:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 16:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 16:53:26,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 16:53:26,635 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:26,636 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 16:53:26,636 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 16:53:27,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 548 ms to try 5000 different iterations.  Found a solution that moves 4730 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17445158001679467. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04120518856326507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05366897751053037, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5255555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04120518856326507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.235, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:27,053 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1040652319:81 , srv1063921129:81 , srv1907734422:82 , srv2079126881:82 , srv639860541:82 , srv827204954:82 , srv12049046:83 , srv1262752896:83 , srv1675559717:83 , srv1777884879:83 , srv307848856:83 , srv446881812:83 , srv763784796:83 , srv911252193:83 , srv1011088674:84 , srv1067512075:84 , srv1246738502:84 , srv1653854934:84 , srv204841728:84 , srv706900250:84 , srv763371005:84 , srv985126142:84 , srv1359379584:85 , srv1479935651:85 , srv1597135585:85 , srv1599668646:85 , srv1640841558:85 , srv1684526744:85 , srv1707092085:85 , srv1955482195:85 , srv2100672922:85 , srv503186585:85 , srv529643373:85 , srv600058728:85 , srv725618115:85 , srv898631494:85 , srv999139865:85 , srv1033891417:86 , srv114612743:86 , srv124620357:86 , srv1342375300:86 , srv1493615921:86 , srv1520785322:86 , srv166389512:86 , srv1689966932:86 , srv1757864304:86 , srv1792874021:86 , srv1890869832:86 , srv1903233800:86 , srv1931865994:86 , srv2032344461:86 , srv203415186:86 , srv252611734:86 , srv370362442:86 , srv4055176:86 , srv73052772:86 , srv824152852:86 , srv893774507:86 , srv968046652:86 , srv1079478287:87 , srv1090164700:87 , srv1147716714:87 , srv1249869182:87 , srv1366339462:87 , srv1564930403:87 , srv1768165078:87 , srv1938840447:87 , srv215586354:87 , srv263034954:87 , srv336682823:87 , srv471985223:87 , srv791717211:87 , srv841148049:87 , srv1161694338:88 , srv1164849861:88 , srv1417857483:88 , srv1780796090:88 , srv223485644:88 , srv320922299:88 , srv363849855:88 , srv472724889:88 , srv672446821:88 , srv790854183:88 , srv918457790:88 , srv932907916:88 , srv998437746:88 , srv1379689369:89 , srv1693713624:89 , srv72578049:89 , srv998354672:89 , srv1785675713:90 , srv1803792158:90 , srv204808782:90 , srv229499859:90 , srv815655821:90 , srv1020399437:92 , srv1610159946:92 , srv2067861830:99 , srv2454807:100 , srv1940280756:456 }
2023-05-16 16:53:27,059 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:27,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1599668646=27, srv1597135585=26, srv2100672922=57, srv998354672=97, srv1033891417=2, srv503186585=73, srv203415186=52, srv363849855=67, srv1520785322=24, srv600058728=75, srv2079126881=56, srv1940280756=49, srv1777884879=39, srv2067861830=55, srv1792874021=42, srv529643373=74, srv73052772=81, srv1780796090=40, srv166389512=31, srv841148049=89, srv1689966932=34, srv1246738502=14, srv999139865=99, srv124620357=13, srv1693713624=35, srv827204954=88, srv1342375300=17, srv229499859=60, srv215586354=58, srv672446821=77, srv791717211=85, srv1161694338=10, srv1684526744=33, srv1785675713=41, srv763371005=82, srv1011088674=0, srv72578049=80, srv1493615921=23, srv204841728=54, srv252611734=62, srv1249869182=15, srv1359379584=18, srv985126142=96, srv1417857483=21, srv1040652319=3, srv1653854934=30, srv263034954=63, srv204808782=53, srv1147716714=9, srv1675559717=32, srv998437746=98, srv1366339462=19, srv1020399437=1, srv893774507=90, srv1757864304=37, srv370362442=68, srv1707092085=36, srv223485644=59, srv2454807=61, srv706900250=78, srv911252193=92, srv1890869832=44, srv4055176=69, srv1768165078=38, srv320922299=65, srv1803792158=43, srv1564930403=25, srv968046652=95, srv898631494=91, srv824152852=87, srv790854183=84, srv1955482195=50, srv1931865994=47, srv815655821=86, srv1610159946=28, srv763784796=83, srv446881812=70, srv932907916=94, srv1640841558=29, srv1379689369=20, srv307848856=64, srv1090164700=7, srv336682823=66, srv12049046=12, srv114612743=8, srv1907734422=46, srv472724889=72, srv639860541=76, srv1063921129=4, srv2032344461=51, srv1938840447=48, srv725618115=79, srv1079478287=6, srv918457790=93, srv1903233800=45, srv1479935651=22, srv1262752896=16, srv1067512075=5, srv471985223=71, srv1164849861=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:27,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:27,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:27,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:27,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 16:53:27,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 16:53:27,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 16:53:27,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 16:53:27,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 16:53:27,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 16:53:27,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 16:53:27,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 16:53:27,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 16:53:27,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04120518856326507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05366897751053037, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04120518856326507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.235, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:27,279 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 16:53:27,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17441845984890314, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04120518856326507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05366897751053037, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04120518856326507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16975, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.235, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 16:53:27,712 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 587 ms to try 5000 different iterations.  Found a solution that moves 2676 regions; Going from a computed imbalance of 0.17441845984890314 to a new imbalance of 0.015603475427139394. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02150468965752568); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030868462107172635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29733333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02150468965752568); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:27,723 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2067861830:72 , srv2454807:72 , srv1020399437:80 , srv204808782:81 , srv229499859:81 , srv1610159946:82 , srv320922299:82 , srv1090164700:83 , srv1366339462:83 , srv1417857483:83 , srv1768165078:83 , srv1803792158:83 , srv215586354:83 , srv72578049:83 , srv815655821:83 , srv932907916:83 , srv1033891417:84 , srv1379689369:84 , srv1520785322:84 , srv1693713624:84 , srv263034954:84 , srv336682823:84 , srv471985223:84 , srv893774507:84 , srv998437746:84 , srv1161694338:85 , srv1164849861:85 , srv1342375300:85 , srv1564930403:85 , srv1640841558:85 , srv1707092085:85 , srv1785675713:85 , srv1903233800:85 , srv1931865994:85 , srv223485644:85 , srv472724889:85 , srv672446821:85 , srv73052772:85 , srv791717211:85 , srv918457790:85 , srv968046652:85 , srv998354672:85 , srv1067512075:86 , srv1079478287:86 , srv114612743:86 , srv1147716714:86 , srv124620357:86 , srv1479935651:86 , srv166389512:86 , srv1689966932:86 , srv1757864304:86 , srv1780796090:86 , srv1938840447:86 , srv2032344461:86 , srv252611734:86 , srv363849855:86 , srv4055176:86 , srv529643373:86 , srv600058728:86 , srv824152852:86 , srv841148049:86 , srv12049046:87 , srv1246738502:87 , srv1262752896:87 , srv1359379584:87 , srv1493615921:87 , srv1653854934:87 , srv1675559717:87 , srv1684526744:87 , srv2100672922:87 , srv370362442:87 , srv446881812:87 , srv503186585:87 , srv790854183:87 , srv827204954:87 , srv1011088674:88 , srv1249869182:88 , srv1597135585:88 , srv1777884879:88 , srv1792874021:88 , srv1890869832:88 , srv1955482195:88 , srv203415186:88 , srv204841728:88 , srv706900250:88 , srv763371005:88 , srv985126142:88 , srv1907734422:89 , srv307848856:89 , srv725618115:89 , srv898631494:89 , srv999139865:89 , srv1063921129:90 , srv911252193:90 , srv639860541:91 , srv763784796:91 , srv1040652319:92 , srv2079126881:92 , srv1599668646:94 , srv1940280756:509 }
2023-05-16 16:53:27,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:27,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1599668646=27, srv1597135585=26, srv2100672922=57, srv998354672=97, srv1033891417=2, srv503186585=73, srv203415186=52, srv363849855=67, srv1520785322=24, srv600058728=75, srv2079126881=56, srv1940280756=49, srv1777884879=39, srv2067861830=55, srv1792874021=42, srv529643373=74, srv73052772=81, srv1780796090=40, srv166389512=31, srv841148049=89, srv1689966932=34, srv1246738502=14, srv999139865=99, srv124620357=13, srv1693713624=35, srv827204954=88, srv1342375300=17, srv229499859=60, srv215586354=58, srv672446821=77, srv791717211=85, srv1161694338=10, srv1684526744=33, srv1785675713=41, srv763371005=82, srv1011088674=0, srv72578049=80, srv1493615921=23, srv204841728=54, srv252611734=62, srv1249869182=15, srv1359379584=18, srv985126142=96, srv1417857483=21, srv1040652319=3, srv1653854934=30, srv263034954=63, srv204808782=53, srv1147716714=9, srv1675559717=32, srv998437746=98, srv1366339462=19, srv1020399437=1, srv893774507=90, srv1757864304=37, srv370362442=68, srv1707092085=36, srv223485644=59, srv2454807=61, srv706900250=78, srv911252193=92, srv1890869832=44, srv4055176=69, srv1768165078=38, srv320922299=65, srv1803792158=43, srv1564930403=25, srv968046652=95, srv898631494=91, srv824152852=87, srv790854183=84, srv1955482195=50, srv1931865994=47, srv815655821=86, srv1610159946=28, srv763784796=83, srv446881812=70, srv932907916=94, srv1640841558=29, srv1379689369=20, srv307848856=64, srv1090164700=7, srv336682823=66, srv12049046=12, srv114612743=8, srv1907734422=46, srv472724889=72, srv639860541=76, srv1063921129=4, srv2032344461=51, srv1938840447=48, srv725618115=79, srv1079478287=6, srv918457790=93, srv1903233800=45, srv1479935651=22, srv1262752896=16, srv1067512075=5, srv471985223=71, srv1164849861=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:27,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:27,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:27,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 16:53:27,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 16:53:27,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 16:53:27,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 16:53:27,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 16:53:27,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 16:53:27,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 16:53:27,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 16:53:27,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 16:53:27,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 16:53:27,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02150468965752568); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030868462107172635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02150468965752568); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:27,885 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 16:53:27,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01558473767464938, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02150468965752568); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030868462107172635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02150468965752568); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 16:53:28,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 1038 regions; Going from a computed imbalance of 0.01558473767464938 to a new imbalance of 8.147746855946251E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006666666666666666); MoveCostFunction : (multiplier=7.0, imbalance=0.11533333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:28,228 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1599668646:78 , srv1249869182:82 , srv1785675713:82 , srv1164849861:83 , srv1890869832:83 , srv1955482195:83 , srv2079126881:83 , srv763784796:83 , srv790854183:83 , srv998354672:83 , srv999139865:83 , srv1147716714:84 , srv1493615921:84 , srv1597135585:84 , srv1693713624:84 , srv1780796090:84 , srv1792874021:84 , srv1803792158:84 , srv203415186:84 , srv223485644:84 , srv363849855:84 , srv898631494:84 , srv1040652319:85 , srv1079478287:85 , srv114612743:85 , srv1161694338:85 , srv124620357:85 , srv1359379584:85 , srv1379689369:85 , srv1610159946:85 , srv1684526744:85 , srv1757864304:85 , srv1938840447:85 , srv2100672922:85 , srv2454807:85 , srv252611734:85 , srv307848856:85 , srv370362442:85 , srv4055176:85 , srv472724889:85 , srv503186585:85 , srv639860541:85 , srv672446821:85 , srv706900250:85 , srv725618115:85 , srv72578049:85 , srv815655821:85 , srv824152852:85 , srv841148049:85 , srv911252193:85 , srv918457790:85 , srv985126142:85 , srv1011088674:86 , srv1020399437:86 , srv1090164700:86 , srv12049046:86 , srv1246738502:86 , srv1417857483:86 , srv1479935651:86 , srv166389512:86 , srv1689966932:86 , srv1777884879:86 , srv1903233800:86 , srv1931865994:86 , srv204808782:86 , srv204841728:86 , srv2067861830:86 , srv471985223:86 , srv529643373:86 , srv600058728:86 , srv73052772:86 , srv763371005:86 , srv791717211:86 , srv932907916:86 , srv968046652:86 , srv1063921129:87 , srv1067512075:87 , srv1262752896:87 , srv1342375300:87 , srv1564930403:87 , srv1640841558:87 , srv1768165078:87 , srv1907734422:87 , srv2032344461:87 , srv229499859:87 , srv263034954:87 , srv336682823:87 , srv446881812:87 , srv998437746:87 , srv1033891417:88 , srv1366339462:88 , srv1520785322:88 , srv1653854934:88 , srv1675559717:88 , srv1707092085:88 , srv215586354:88 , srv320922299:88 , srv893774507:88 , srv827204954:89 , srv1940280756:543 }
2023-05-16 16:53:28,232 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:28,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1599668646=27, srv1597135585=26, srv2100672922=57, srv998354672=97, srv1033891417=2, srv503186585=73, srv203415186=52, srv363849855=67, srv1520785322=24, srv600058728=75, srv2079126881=56, srv1940280756=49, srv1777884879=39, srv2067861830=55, srv1792874021=42, srv529643373=74, srv73052772=81, srv1780796090=40, srv166389512=31, srv841148049=89, srv1689966932=34, srv1246738502=14, srv999139865=99, srv124620357=13, srv1693713624=35, srv827204954=88, srv1342375300=17, srv229499859=60, srv215586354=58, srv672446821=77, srv791717211=85, srv1161694338=10, srv1684526744=33, srv1785675713=41, srv763371005=82, srv1011088674=0, srv72578049=80, srv1493615921=23, srv204841728=54, srv252611734=62, srv1249869182=15, srv1359379584=18, srv985126142=96, srv1417857483=21, srv1040652319=3, srv1653854934=30, srv263034954=63, srv204808782=53, srv1147716714=9, srv1675559717=32, srv998437746=98, srv1366339462=19, srv1020399437=1, srv893774507=90, srv1757864304=37, srv370362442=68, srv1707092085=36, srv223485644=59, srv2454807=61, srv706900250=78, srv911252193=92, srv1890869832=44, srv4055176=69, srv1768165078=38, srv320922299=65, srv1803792158=43, srv1564930403=25, srv968046652=95, srv898631494=91, srv824152852=87, srv790854183=84, srv1955482195=50, srv1931865994=47, srv815655821=86, srv1610159946=28, srv763784796=83, srv446881812=70, srv932907916=94, srv1640841558=29, srv1379689369=20, srv307848856=64, srv1090164700=7, srv336682823=66, srv12049046=12, srv114612743=8, srv1907734422=46, srv472724889=72, srv639860541=76, srv1063921129=4, srv2032344461=51, srv1938840447=48, srv725618115=79, srv1079478287=6, srv918457790=93, srv1903233800=45, srv1479935651=22, srv1262752896=16, srv1067512075=5, srv471985223=71, srv1164849861=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:28,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:28,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 16:53:28,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 16:53:28,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 16:53:28,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 16:53:28,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 16:53:28,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 16:53:28,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 16:53:28,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006666666666666666); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:28,410 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-16 16:53:28,410 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.075064542476015E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006666666666666666); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-16 16:53:28,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 536 ms to try 5000 different iterations.  Found a solution that moves 265 regions; Going from a computed imbalance of 8.075064542476015E-4 to a new imbalance of 2.1991144620428267E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.029444444444444443, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:28,810 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1020399437:83 , srv1366339462:83 , srv1564930403:83 , srv1610159946:83 , srv1653854934:83 , srv725618115:83 , srv918457790:83 , srv998437746:83 , srv1011088674:84 , srv1040652319:84 , srv1063921129:84 , srv1067512075:84 , srv1079478287:84 , srv1147716714:84 , srv1161694338:84 , srv1342375300:84 , srv1379689369:84 , srv1479935651:84 , srv1520785322:84 , srv1597135585:84 , srv166389512:84 , srv1675559717:84 , srv1689966932:84 , srv1707092085:84 , srv1757864304:84 , srv1768165078:84 , srv1780796090:84 , srv1792874021:84 , srv1907734422:84 , srv1938840447:84 , srv2032344461:84 , srv203415186:84 , srv204841728:84 , srv215586354:84 , srv229499859:84 , srv263034954:84 , srv320922299:84 , srv336682823:84 , srv370362442:84 , srv472724889:84 , srv529643373:84 , srv672446821:84 , srv763371005:84 , srv790854183:84 , srv791717211:84 , srv815655821:84 , srv827204954:84 , srv893774507:84 , srv898631494:84 , srv911252193:84 , srv1033891417:85 , srv114612743:85 , srv124620357:85 , srv1246738502:85 , srv1262752896:85 , srv1359379584:85 , srv1417857483:85 , srv1493615921:85 , srv1599668646:85 , srv1640841558:85 , srv1684526744:85 , srv1777884879:85 , srv1803792158:85 , srv1890869832:85 , srv1903233800:85 , srv1931865994:85 , srv204808782:85 , srv2067861830:85 , srv2079126881:85 , srv2100672922:85 , srv2454807:85 , srv252611734:85 , srv363849855:85 , srv4055176:85 , srv446881812:85 , srv471985223:85 , srv503186585:85 , srv600058728:85 , srv639860541:85 , srv706900250:85 , srv72578049:85 , srv73052772:85 , srv763784796:85 , srv824152852:85 , srv841148049:85 , srv932907916:85 , srv968046652:85 , srv985126142:85 , srv998354672:85 , srv999139865:85 , srv1090164700:86 , srv1164849861:86 , srv12049046:86 , srv1249869182:86 , srv1693713624:86 , srv1785675713:86 , srv1955482195:86 , srv223485644:86 , srv307848856:86 , srv1940280756:634 }
2023-05-16 16:53:28,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:28,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1599668646=27, srv1597135585=26, srv2100672922=57, srv998354672=97, srv1033891417=2, srv503186585=73, srv203415186=52, srv363849855=67, srv1520785322=24, srv600058728=75, srv2079126881=56, srv1940280756=49, srv1777884879=39, srv2067861830=55, srv1792874021=42, srv529643373=74, srv73052772=81, srv1780796090=40, srv166389512=31, srv841148049=89, srv1689966932=34, srv1246738502=14, srv999139865=99, srv124620357=13, srv1693713624=35, srv827204954=88, srv1342375300=17, srv229499859=60, srv215586354=58, srv672446821=77, srv791717211=85, srv1161694338=10, srv1684526744=33, srv1785675713=41, srv763371005=82, srv1011088674=0, srv72578049=80, srv1493615921=23, srv204841728=54, srv252611734=62, srv1249869182=15, srv1359379584=18, srv985126142=96, srv1417857483=21, srv1040652319=3, srv1653854934=30, srv263034954=63, srv204808782=53, srv1147716714=9, srv1675559717=32, srv998437746=98, srv1366339462=19, srv1020399437=1, srv893774507=90, srv1757864304=37, srv370362442=68, srv1707092085=36, srv223485644=59, srv2454807=61, srv706900250=78, srv911252193=92, srv1890869832=44, srv4055176=69, srv1768165078=38, srv320922299=65, srv1803792158=43, srv1564930403=25, srv968046652=95, srv898631494=91, srv824152852=87, srv790854183=84, srv1955482195=50, srv1931865994=47, srv815655821=86, srv1610159946=28, srv763784796=83, srv446881812=70, srv932907916=94, srv1640841558=29, srv1379689369=20, srv307848856=64, srv1090164700=7, srv336682823=66, srv12049046=12, srv114612743=8, srv1907734422=46, srv472724889=72, srv639860541=76, srv1063921129=4, srv2032344461=51, srv1938840447=48, srv725618115=79, srv1079478287=6, srv918457790=93, srv1903233800=45, srv1479935651=22, srv1262752896=16, srv1067512075=5, srv471985223=71, srv1164849861=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:28,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:28,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-16 16:53:28,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-16 16:53:28,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-16 16:53:28,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-16 16:53:28,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-16 16:53:28,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-16 16:53:28,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-16 16:53:28,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-16 16:53:28,989 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.013557496054268E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:28,989 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1020399437:83 , srv1366339462:83 , srv1564930403:83 , srv1610159946:83 , srv1653854934:83 , srv725618115:83 , srv918457790:83 , srv998437746:83 , srv1011088674:84 , srv1040652319:84 , srv1063921129:84 , srv1067512075:84 , srv1079478287:84 , srv1147716714:84 , srv1161694338:84 , srv1342375300:84 , srv1379689369:84 , srv1479935651:84 , srv1520785322:84 , srv1597135585:84 , srv166389512:84 , srv1675559717:84 , srv1689966932:84 , srv1707092085:84 , srv1757864304:84 , srv1768165078:84 , srv1780796090:84 , srv1792874021:84 , srv1907734422:84 , srv1938840447:84 , srv2032344461:84 , srv203415186:84 , srv204841728:84 , srv215586354:84 , srv229499859:84 , srv263034954:84 , srv320922299:84 , srv336682823:84 , srv370362442:84 , srv472724889:84 , srv529643373:84 , srv672446821:84 , srv763371005:84 , srv790854183:84 , srv791717211:84 , srv815655821:84 , srv827204954:84 , srv893774507:84 , srv898631494:84 , srv911252193:84 , srv1033891417:85 , srv114612743:85 , srv124620357:85 , srv1246738502:85 , srv1262752896:85 , srv1359379584:85 , srv1417857483:85 , srv1493615921:85 , srv1599668646:85 , srv1640841558:85 , srv1684526744:85 , srv1777884879:85 , srv1803792158:85 , srv1890869832:85 , srv1903233800:85 , srv1931865994:85 , srv204808782:85 , srv2067861830:85 , srv2079126881:85 , srv2100672922:85 , srv2454807:85 , srv252611734:85 , srv363849855:85 , srv4055176:85 , srv446881812:85 , srv471985223:85 , srv503186585:85 , srv600058728:85 , srv639860541:85 , srv706900250:85 , srv72578049:85 , srv73052772:85 , srv763784796:85 , srv824152852:85 , srv841148049:85 , srv932907916:85 , srv968046652:85 , srv985126142:85 , srv998354672:85 , srv999139865:85 , srv1090164700:86 , srv1164849861:86 , srv12049046:86 , srv1249869182:86 , srv1693713624:86 , srv1785675713:86 , srv1955482195:86 , srv223485644:86 , srv307848856:86 , srv1940280756:634 }
2023-05-16 16:53:29,040 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=415 (was 415), ProcessCount=176 (was 176), AvailableMemoryMB=11060 (was 11897)
2023-05-16 16:53:29,053 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=415, ProcessCount=176, AvailableMemoryMB=11060
2023-05-16 16:53:29,054 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-16 16:53:29,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-16 16:53:29,066 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1065037006:84 , srv1080335239:84 , srv1122161815:84 , srv1123600565:84 , srv1232508571:84 , srv1263239789:84 , srv1341265140:84 , srv1443881237:84 , srv1498560184:84 , srv1628571848:84 , srv1644027342:84 , srv1764882787:84 , srv1770297548:84 , srv1808831529:84 , srv1811341844:84 , srv1824122261:84 , srv1925076045:84 , srv194961307:84 , srv2041330444:84 , srv2052742595:84 , srv228427243:84 , srv275845810:84 , srv289466075:84 , srv32409503:84 , srv325485017:84 , srv406084678:84 , srv481037615:84 , srv671590329:84 , srv915186319:84 , srv1885380149:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-16 16:53:29,068 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:53:29,069 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1498560184=8, srv1885380149=16, srv1811341844=14, srv406084678=26, srv1080335239=1, srv1263239789=5, srv325485017=25, srv1123600565=3, srv481037615=27, srv289466075=23, srv915186319=29, srv275845810=22, srv1644027342=10, srv2041330444=19, srv1770297548=12, srv1824122261=15, srv194961307=18, srv1628571848=9, srv671590329=28, srv1122161815=2, srv1443881237=7, srv228427243=21, srv1065037006=0, srv1341265140=6, srv1232508571=4, srv1764882787=11, srv32409503=24, srv2052742595=20, srv1808831529=13, srv1925076045=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:53:29,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:53:29,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:53:29,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:53:29,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:53:29,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:53:29,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:53:29,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:53:29,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:53:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-16 16:53:29,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1562295551059936, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:53:29,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912685317491053, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1562295551059936, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-16 16:55:29,099 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6123447 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912685317491053 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:55:29,106 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1065037006:90 , srv1080335239:90 , srv1122161815:90 , srv1123600565:90 , srv1232508571:90 , srv1263239789:90 , srv1341265140:90 , srv1443881237:90 , srv1498560184:90 , srv1628571848:90 , srv1644027342:90 , srv1764882787:90 , srv1770297548:90 , srv1808831529:90 , srv1811341844:90 , srv1824122261:90 , srv1885380149:90 , srv1925076045:90 , srv194961307:90 , srv2041330444:90 , srv2052742595:90 , srv228427243:90 , srv275845810:90 , srv289466075:90 , srv32409503:90 , srv325485017:90 , srv406084678:90 , srv481037615:90 , srv671590329:90 , srv915186319:90 }
2023-05-16 16:55:29,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-16 16:55:29,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1498560184=8, srv1885380149=16, srv1811341844=14, srv406084678=26, srv1080335239=1, srv1263239789=5, srv325485017=25, srv1123600565=3, srv481037615=27, srv289466075=23, srv915186319=29, srv275845810=22, srv1644027342=10, srv2041330444=19, srv1770297548=12, srv1824122261=15, srv194961307=18, srv1628571848=9, srv671590329=28, srv1122161815=2, srv1443881237=7, srv228427243=21, srv1065037006=0, srv1341265140=6, srv1232508571=4, srv1764882787=11, srv32409503=24, srv2052742595=20, srv1808831529=13, srv1925076045=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-16 16:55:29,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-16 16:55:29,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-16 16:55:29,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-16 16:55:29,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-16 16:55:29,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-16 16:55:29,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-16 16:55:29,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-16 16:55:29,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-16 16:55:29,125 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1065037006:90 , srv1080335239:90 , srv1122161815:90 , srv1123600565:90 , srv1232508571:90 , srv1263239789:90 , srv1341265140:90 , srv1443881237:90 , srv1498560184:90 , srv1628571848:90 , srv1644027342:90 , srv1764882787:90 , srv1770297548:90 , srv1808831529:90 , srv1811341844:90 , srv1824122261:90 , srv1885380149:90 , srv1925076045:90 , srv194961307:90 , srv2041330444:90 , srv2052742595:90 , srv228427243:90 , srv275845810:90 , srv289466075:90 , srv32409503:90 , srv325485017:90 , srv406084678:90 , srv481037615:90 , srv671590329:90 , srv915186319:90 }
2023-05-16 16:55:29,140 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=204 (was 415), ProcessCount=171 (was 176), AvailableMemoryMB=11458 (was 11060) - AvailableMemoryMB LEAK? -