2023-03-20 00:58:18,117 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-20 00:58:18,436 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-20 00:58:18,546 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-20 00:58:18,613 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 00:58:18,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-20 00:58:18,629 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=520, ProcessCount=166, AvailableMemoryMB=5471
2023-03-20 00:58:18,632 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 00:58:18,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-20 00:58:18,755 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1016460172:84 , srv1019606751:84 , srv10214305:84 , srv1169317714:84 , srv1181103591:84 , srv1186689076:84 , srv1220757353:84 , srv1222416459:84 , srv1224136500:84 , srv1239163918:84 , srv123930359:84 , srv1280533394:84 , srv1291188301:84 , srv1392911281:84 , srv1416700660:84 , srv1431851029:84 , srv1485996259:84 , srv1521227373:84 , srv1529394336:84 , srv1535448962:84 , srv1591553004:84 , srv164747465:84 , srv1652899992:84 , srv1672314092:84 , srv1677797902:84 , srv1687117649:84 , srv1706273526:84 , srv1726679887:84 , srv1731612342:84 , srv1740207560:84 , srv1754991461:84 , srv1758688337:84 , srv1779848597:84 , srv1785313335:84 , srv1818552263:84 , srv1829170310:84 , srv1829919426:84 , srv1847267499:84 , srv185040309:84 , srv1853418669:84 , srv1880039617:84 , srv1929520444:84 , srv1958398779:84 , srv198002107:84 , srv2001602751:84 , srv2026405032:84 , srv2030316924:84 , srv205255585:84 , srv2111204854:84 , srv2134166520:84 , srv2136917873:84 , srv219693326:84 , srv244051217:84 , srv270652197:84 , srv294333900:84 , srv305894910:84 , srv32379879:84 , srv326089414:84 , srv334625049:84 , srv348340156:84 , srv373013607:84 , srv373106243:84 , srv375656478:84 , srv379992304:84 , srv406621625:84 , srv497026709:84 , srv549922518:84 , srv550992595:84 , srv587867855:84 , srv597266931:84 , srv625985471:84 , srv626492456:84 , srv626537085:84 , srv62706972:84 , srv669073724:84 , srv67742609:84 , srv693570846:84 , srv694482373:84 , srv724863097:84 , srv733315781:84 , srv752480339:84 , srv777089497:84 , srv782473235:84 , srv829973161:84 , srv852353570:84 , srv868987788:84 , srv869933052:84 , srv877267438:84 , srv890365972:84 , srv911022551:84 , srv920901843:84 , srv930587663:84 , srv930902737:84 , srv937112699:84 , srv946652928:84 , srv947222909:84 , srv962542564:84 , srv975813180:84 , srv991763763:84 , srv621635165:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-20 00:58:18,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:18,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv946652928=95, srv10214305=2, srv123930359=10, srv852353570=85, srv587867855=68, srv334625049=58, srv1169317714=3, srv869933052=87, srv1181103591=4, srv326089414=57, srv868987788=86, srv1224136500=8, srv947222909=96, srv373013607=60, srv1740207560=29, srv1929520444=41, srv1222416459=7, srv32379879=56, srv626537085=73, srv733315781=80, srv1677797902=24, srv1758688337=31, srv1785313335=33, srv991763763=99, srv1016460172=0, srv67742609=76, srv373106243=61, srv1239163918=9, srv244051217=52, srv1591553004=20, srv669073724=75, srv1880039617=40, srv1186689076=5, srv930902737=93, srv2136917873=50, srv625985471=71, srv937112699=94, srv270652197=53, srv890365972=89, srv1521227373=17, srv497026709=65, srv305894910=55, srv724863097=79, srv1829170310=35, srv294333900=54, srv375656478=62, srv1431851029=15, srv1706273526=26, srv693570846=77, srv752480339=81, srv1291188301=12, srv1392911281=13, srv1847267499=37, srv777089497=82, srv1829919426=36, srv1958398779=42, srv962542564=97, srv2001602751=44, srv406621625=64, srv1652899992=22, srv1818552263=34, srv219693326=51, srv621635165=70, srv1726679887=27, srv626492456=72, srv920901843=91, srv1220757353=6, srv164747465=21, srv198002107=43, srv911022551=90, srv185040309=38, srv829973161=84, srv597266931=69, srv550992595=67, srv1416700660=14, srv1019606751=1, srv1485996259=16, srv205255585=47, srv348340156=59, srv1853418669=39, srv1779848597=32, srv782473235=83, srv2134166520=49, srv930587663=92, srv1687117649=25, srv2030316924=46, srv1672314092=23, srv379992304=63, srv549922518=66, srv62706972=74, srv694482373=78, srv1535448962=19, srv975813180=98, srv1731612342=28, srv877267438=88, srv1529394336=18, srv2111204854=48, srv1280533394=11, srv2026405032=45, srv1754991461=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:18,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:18,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 00:58:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 00:58:18,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 00:58:18,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 00:58:18,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 00:58:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 00:58:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 00:58:18,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 00:58:18,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 00:58:18,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 00:58:19,006 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:19,006 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 00:58:19,006 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 00:58:19,587 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 713 ms to try 5000 different iterations.  Found a solution that moves 4677 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1829889837749651. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03861261492925301, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647720356929738, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5196666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03861261492925301, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:19,607 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv947222909:81 , srv1291188301:82 , srv1416700660:82 , srv1731612342:82 , srv32379879:82 , srv373106243:82 , srv930587663:82 , srv1687117649:83 , srv379992304:83 , srv626492456:83 , srv67742609:83 , srv829973161:83 , srv1224136500:84 , srv1392911281:84 , srv1740207560:84 , srv1785313335:84 , srv1818552263:84 , srv406621625:84 , srv62706972:84 , srv877267438:84 , srv890365972:84 , srv930902737:84 , srv1016460172:85 , srv1239163918:85 , srv1485996259:85 , srv1706273526:85 , srv1758688337:85 , srv1829170310:85 , srv1829919426:85 , srv1929520444:85 , srv305894910:85 , srv375656478:85 , srv550992595:85 , srv587867855:85 , srv752480339:85 , srv920901843:85 , srv1181103591:86 , srv1222416459:86 , srv1535448962:86 , srv1652899992:86 , srv1779848597:86 , srv1847267499:86 , srv185040309:86 , srv1958398779:86 , srv198002107:86 , srv2001602751:86 , srv2111204854:86 , srv2134166520:86 , srv219693326:86 , srv244051217:86 , srv497026709:86 , srv597266931:86 , srv869933052:86 , srv911022551:86 , srv1169317714:87 , srv1186689076:87 , srv123930359:87 , srv1280533394:87 , srv1726679887:87 , srv1754991461:87 , srv1853418669:87 , srv1880039617:87 , srv205255585:87 , srv348340156:87 , srv373013607:87 , srv782473235:87 , srv1220757353:88 , srv1591553004:88 , srv270652197:88 , srv625985471:88 , srv693570846:88 , srv724863097:88 , srv733315781:88 , srv946652928:88 , srv991763763:88 , srv1019606751:89 , srv10214305:89 , srv1521227373:89 , srv1529394336:89 , srv2030316924:89 , srv294333900:89 , srv326089414:89 , srv626537085:89 , srv669073724:89 , srv852353570:89 , srv962542564:89 , srv1672314092:90 , srv1677797902:90 , srv2026405032:90 , srv2136917873:90 , srv868987788:90 , srv937112699:90 , srv975813180:90 , srv334625049:92 , srv549922518:92 , srv694482373:92 , srv1431851029:93 , srv164747465:93 , srv777089497:94 , srv621635165:433 }
2023-03-20 00:58:19,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:19,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv946652928=95, srv10214305=2, srv123930359=10, srv852353570=85, srv587867855=68, srv334625049=58, srv1169317714=3, srv869933052=87, srv1181103591=4, srv326089414=57, srv868987788=86, srv1224136500=8, srv947222909=96, srv373013607=60, srv1740207560=29, srv1929520444=41, srv1222416459=7, srv32379879=56, srv626537085=73, srv733315781=80, srv1677797902=24, srv1758688337=31, srv1785313335=33, srv991763763=99, srv1016460172=0, srv67742609=76, srv373106243=61, srv1239163918=9, srv244051217=52, srv1591553004=20, srv669073724=75, srv1880039617=40, srv1186689076=5, srv930902737=93, srv2136917873=50, srv625985471=71, srv937112699=94, srv270652197=53, srv890365972=89, srv1521227373=17, srv497026709=65, srv305894910=55, srv724863097=79, srv1829170310=35, srv294333900=54, srv375656478=62, srv1431851029=15, srv1706273526=26, srv693570846=77, srv752480339=81, srv1291188301=12, srv1392911281=13, srv1847267499=37, srv777089497=82, srv1829919426=36, srv1958398779=42, srv962542564=97, srv2001602751=44, srv406621625=64, srv1652899992=22, srv1818552263=34, srv219693326=51, srv621635165=70, srv1726679887=27, srv626492456=72, srv920901843=91, srv1220757353=6, srv164747465=21, srv198002107=43, srv911022551=90, srv185040309=38, srv829973161=84, srv597266931=69, srv550992595=67, srv1416700660=14, srv1019606751=1, srv1485996259=16, srv205255585=47, srv348340156=59, srv1853418669=39, srv1779848597=32, srv782473235=83, srv2134166520=49, srv930587663=92, srv1687117649=25, srv2030316924=46, srv1672314092=23, srv379992304=63, srv549922518=66, srv62706972=74, srv694482373=78, srv1535448962=19, srv975813180=98, srv1731612342=28, srv877267438=88, srv1529394336=18, srv2111204854=48, srv1280533394=11, srv2026405032=45, srv1754991461=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:19,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:19,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:19,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 00:58:19,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 00:58:19,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 00:58:19,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 00:58:19,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 00:58:19,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 00:58:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 00:58:19,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 00:58:19,733 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03861261492925301, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647720356929738, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03861261492925301, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:19,733 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 00:58:19,733 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18295623472100556, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03861261492925301, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05647720356929738, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03861261492925301, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17816666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24566666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 00:58:19,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 2631 regions; Going from a computed imbalance of 0.18295623472100556 to a new imbalance of 0.016577430604640127. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034934603004659344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29233333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.046, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:19,994 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1431851029:79 , srv777089497:79 , srv164747465:80 , srv1677797902:80 , srv334625049:80 , srv1019606751:81 , srv10214305:81 , srv1672314092:81 , srv2136917873:81 , srv549922518:81 , srv937112699:82 , srv1169317714:83 , srv2026405032:83 , srv2030316924:83 , srv294333900:83 , srv326089414:83 , srv348340156:83 , srv497026709:83 , srv625985471:83 , srv669073724:83 , srv694482373:83 , srv733315781:83 , srv868987788:83 , srv946652928:83 , srv962542564:83 , srv975813180:83 , srv991763763:83 , srv1220757353:84 , srv1521227373:84 , srv1535448962:84 , srv1847267499:84 , srv1880039617:84 , srv205255585:84 , srv2111204854:84 , srv219693326:84 , srv244051217:84 , srv270652197:84 , srv626537085:84 , srv782473235:84 , srv852353570:84 , srv1222416459:85 , srv1280533394:85 , srv1591553004:85 , srv1726679887:85 , srv1754991461:85 , srv185040309:85 , srv1853418669:85 , srv198002107:85 , srv2001602751:85 , srv305894910:85 , srv375656478:85 , srv1181103591:86 , srv1186689076:86 , srv1239163918:86 , srv123930359:86 , srv1529394336:86 , srv1779848597:86 , srv1929520444:86 , srv1958398779:86 , srv373013607:86 , srv587867855:86 , srv693570846:86 , srv752480339:86 , srv1392911281:87 , srv1485996259:87 , srv1758688337:87 , srv1829170310:87 , srv1829919426:87 , srv2134166520:87 , srv373106243:87 , srv62706972:87 , srv724863097:87 , srv911022551:87 , srv1016460172:88 , srv1224136500:88 , srv1652899992:88 , srv1687117649:88 , srv1706273526:88 , srv1785313335:88 , srv406621625:88 , srv550992595:88 , srv597266931:88 , srv920901843:88 , srv1291188301:89 , srv1818552263:89 , srv379992304:89 , srv67742609:89 , srv890365972:89 , srv930587663:89 , srv930902737:89 , srv1740207560:90 , srv626492456:90 , srv829973161:90 , srv877267438:90 , srv1416700660:91 , srv1731612342:91 , srv32379879:91 , srv869933052:91 , srv947222909:93 , srv621635165:538 }
2023-03-20 00:58:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv946652928=95, srv10214305=2, srv123930359=10, srv852353570=85, srv587867855=68, srv334625049=58, srv1169317714=3, srv869933052=87, srv1181103591=4, srv326089414=57, srv868987788=86, srv1224136500=8, srv947222909=96, srv373013607=60, srv1740207560=29, srv1929520444=41, srv1222416459=7, srv32379879=56, srv626537085=73, srv733315781=80, srv1677797902=24, srv1758688337=31, srv1785313335=33, srv991763763=99, srv1016460172=0, srv67742609=76, srv373106243=61, srv1239163918=9, srv244051217=52, srv1591553004=20, srv669073724=75, srv1880039617=40, srv1186689076=5, srv930902737=93, srv2136917873=50, srv625985471=71, srv937112699=94, srv270652197=53, srv890365972=89, srv1521227373=17, srv497026709=65, srv305894910=55, srv724863097=79, srv1829170310=35, srv294333900=54, srv375656478=62, srv1431851029=15, srv1706273526=26, srv693570846=77, srv752480339=81, srv1291188301=12, srv1392911281=13, srv1847267499=37, srv777089497=82, srv1829919426=36, srv1958398779=42, srv962542564=97, srv2001602751=44, srv406621625=64, srv1652899992=22, srv1818552263=34, srv219693326=51, srv621635165=70, srv1726679887=27, srv626492456=72, srv920901843=91, srv1220757353=6, srv164747465=21, srv198002107=43, srv911022551=90, srv185040309=38, srv829973161=84, srv597266931=69, srv550992595=67, srv1416700660=14, srv1019606751=1, srv1485996259=16, srv205255585=47, srv348340156=59, srv1853418669=39, srv1779848597=32, srv782473235=83, srv2134166520=49, srv930587663=92, srv1687117649=25, srv2030316924=46, srv1672314092=23, srv379992304=63, srv549922518=66, srv62706972=74, srv694482373=78, srv1535448962=19, srv975813180=98, srv1731612342=28, srv877267438=88, srv1529394336=18, srv2111204854=48, srv1280533394=11, srv2026405032=45, srv1754991461=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:20,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 00:58:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 00:58:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 00:58:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 00:58:20,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 00:58:20,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 00:58:20,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 00:58:20,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 00:58:20,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 00:58:20,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 00:58:20,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 00:58:20,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 00:58:20,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034934603004659344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.046, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,155 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 00:58:20,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01655900794888481, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034934603004659344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.046, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 00:58:20,312 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 263 ms to try 5000 different iterations.  Found a solution that moves 1016 regions; Going from a computed imbalance of 0.01655900794888481 to a new imbalance of 9.949669281104324E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008347724199692328); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012190040275996437); MoveCostFunction : (multiplier=7.0, imbalance=0.11288888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008347724199692328); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,313 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv869933052:80 , srv694482373:82 , srv947222909:82 , srv1529394336:83 , srv1652899992:83 , srv724863097:83 , srv829973161:83 , srv868987788:83 , srv877267438:83 , srv890365972:83 , srv1016460172:84 , srv1186689076:84 , srv123930359:84 , srv1416700660:84 , srv1731612342:84 , srv1740207560:84 , srv1758688337:84 , srv1818552263:84 , srv373013607:84 , srv549922518:84 , srv550992595:84 , srv597266931:84 , srv626492456:84 , srv669073724:84 , srv693570846:84 , srv777089497:84 , srv852353570:84 , srv920901843:84 , srv930902737:84 , srv975813180:84 , srv1181103591:85 , srv1220757353:85 , srv1224136500:85 , srv1280533394:85 , srv1291188301:85 , srv1431851029:85 , srv1521227373:85 , srv1591553004:85 , srv164747465:85 , srv1706273526:85 , srv1785313335:85 , srv1829170310:85 , srv1853418669:85 , srv1929520444:85 , srv1958398779:85 , srv198002107:85 , srv2026405032:85 , srv32379879:85 , srv406621625:85 , srv587867855:85 , srv626537085:85 , srv67742609:85 , srv733315781:85 , srv782473235:85 , srv911022551:85 , srv930587663:85 , srv962542564:85 , srv10214305:86 , srv1169317714:86 , srv1239163918:86 , srv1392911281:86 , srv1485996259:86 , srv1672314092:86 , srv1726679887:86 , srv1754991461:86 , srv1779848597:86 , srv1880039617:86 , srv2030316924:86 , srv205255585:86 , srv2134166520:86 , srv270652197:86 , srv294333900:86 , srv305894910:86 , srv326089414:86 , srv334625049:86 , srv379992304:86 , srv752480339:86 , srv937112699:86 , srv991763763:86 , srv1019606751:87 , srv1222416459:87 , srv1677797902:87 , srv1687117649:87 , srv1829919426:87 , srv185040309:87 , srv2001602751:87 , srv2111204854:87 , srv2136917873:87 , srv219693326:87 , srv244051217:87 , srv625985471:87 , srv62706972:87 , srv946652928:87 , srv1535448962:88 , srv1847267499:88 , srv373106243:88 , srv375656478:88 , srv497026709:88 , srv348340156:89 , srv621635165:561 }
2023-03-20 00:58:20,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:20,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv946652928=95, srv10214305=2, srv123930359=10, srv852353570=85, srv587867855=68, srv334625049=58, srv1169317714=3, srv869933052=87, srv1181103591=4, srv326089414=57, srv868987788=86, srv1224136500=8, srv947222909=96, srv373013607=60, srv1740207560=29, srv1929520444=41, srv1222416459=7, srv32379879=56, srv626537085=73, srv733315781=80, srv1677797902=24, srv1758688337=31, srv1785313335=33, srv991763763=99, srv1016460172=0, srv67742609=76, srv373106243=61, srv1239163918=9, srv244051217=52, srv1591553004=20, srv669073724=75, srv1880039617=40, srv1186689076=5, srv930902737=93, srv2136917873=50, srv625985471=71, srv937112699=94, srv270652197=53, srv890365972=89, srv1521227373=17, srv497026709=65, srv305894910=55, srv724863097=79, srv1829170310=35, srv294333900=54, srv375656478=62, srv1431851029=15, srv1706273526=26, srv693570846=77, srv752480339=81, srv1291188301=12, srv1392911281=13, srv1847267499=37, srv777089497=82, srv1829919426=36, srv1958398779=42, srv962542564=97, srv2001602751=44, srv406621625=64, srv1652899992=22, srv1818552263=34, srv219693326=51, srv621635165=70, srv1726679887=27, srv626492456=72, srv920901843=91, srv1220757353=6, srv164747465=21, srv198002107=43, srv911022551=90, srv185040309=38, srv829973161=84, srv597266931=69, srv550992595=67, srv1416700660=14, srv1019606751=1, srv1485996259=16, srv205255585=47, srv348340156=59, srv1853418669=39, srv1779848597=32, srv782473235=83, srv2134166520=49, srv930587663=92, srv1687117649=25, srv2030316924=46, srv1672314092=23, srv379992304=63, srv549922518=66, srv62706972=74, srv694482373=78, srv1535448962=19, srv975813180=98, srv1731612342=28, srv877267438=88, srv1529394336=18, srv2111204854=48, srv1280533394=11, srv2026405032=45, srv1754991461=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:20,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:20,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 00:58:20,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 00:58:20,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 00:58:20,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 00:58:20,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 00:58:20,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 00:58:20,415 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008347724199692328); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012190040275996437); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008347724199692328); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,416 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-20 00:58:20,416 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.878527440559275E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008347724199692328); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012190040275996437); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008347724199692328); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-20 00:58:20,614 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 275 ms to try 5000 different iterations.  Found a solution that moves 310 regions; Going from a computed imbalance of 9.878527440559275E-4 to a new imbalance of 1.3231273671866872E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.034444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,615 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv348340156:82 , srv1687117649:83 , srv1829919426:83 , srv2134166520:83 , srv334625049:83 , srv1222416459:84 , srv123930359:84 , srv1485996259:84 , srv1521227373:84 , srv1529394336:84 , srv1535448962:84 , srv1591553004:84 , srv164747465:84 , srv1706273526:84 , srv1726679887:84 , srv1740207560:84 , srv1754991461:84 , srv1779848597:84 , srv1829170310:84 , srv1847267499:84 , srv185040309:84 , srv2001602751:84 , srv2026405032:84 , srv2030316924:84 , srv2136917873:84 , srv270652197:84 , srv294333900:84 , srv32379879:84 , srv326089414:84 , srv379992304:84 , srv597266931:84 , srv625985471:84 , srv626537085:84 , srv62706972:84 , srv693570846:84 , srv724863097:84 , srv911022551:84 , srv937112699:84 , srv946652928:84 , srv1016460172:85 , srv1019606751:85 , srv1186689076:85 , srv1220757353:85 , srv1224136500:85 , srv1239163918:85 , srv1280533394:85 , srv1416700660:85 , srv1431851029:85 , srv1652899992:85 , srv1672314092:85 , srv1677797902:85 , srv1731612342:85 , srv1785313335:85 , srv1818552263:85 , srv1853418669:85 , srv1880039617:85 , srv1958398779:85 , srv205255585:85 , srv2111204854:85 , srv219693326:85 , srv244051217:85 , srv373013607:85 , srv373106243:85 , srv375656478:85 , srv406621625:85 , srv549922518:85 , srv550992595:85 , srv626492456:85 , srv67742609:85 , srv694482373:85 , srv752480339:85 , srv777089497:85 , srv868987788:85 , srv869933052:85 , srv877267438:85 , srv920901843:85 , srv930902737:85 , srv947222909:85 , srv962542564:85 , srv991763763:85 , srv10214305:86 , srv1169317714:86 , srv1181103591:86 , srv1291188301:86 , srv1392911281:86 , srv1758688337:86 , srv1929520444:86 , srv198002107:86 , srv305894910:86 , srv497026709:86 , srv587867855:86 , srv669073724:86 , srv733315781:86 , srv782473235:86 , srv829973161:86 , srv852353570:86 , srv930587663:86 , srv975813180:86 , srv890365972:87 , srv621635165:610 }
2023-03-20 00:58:20,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:20,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv946652928=95, srv10214305=2, srv123930359=10, srv852353570=85, srv587867855=68, srv334625049=58, srv1169317714=3, srv869933052=87, srv1181103591=4, srv326089414=57, srv868987788=86, srv1224136500=8, srv947222909=96, srv373013607=60, srv1740207560=29, srv1929520444=41, srv1222416459=7, srv32379879=56, srv626537085=73, srv733315781=80, srv1677797902=24, srv1758688337=31, srv1785313335=33, srv991763763=99, srv1016460172=0, srv67742609=76, srv373106243=61, srv1239163918=9, srv244051217=52, srv1591553004=20, srv669073724=75, srv1880039617=40, srv1186689076=5, srv930902737=93, srv2136917873=50, srv625985471=71, srv937112699=94, srv270652197=53, srv890365972=89, srv1521227373=17, srv497026709=65, srv305894910=55, srv724863097=79, srv1829170310=35, srv294333900=54, srv375656478=62, srv1431851029=15, srv1706273526=26, srv693570846=77, srv752480339=81, srv1291188301=12, srv1392911281=13, srv1847267499=37, srv777089497=82, srv1829919426=36, srv1958398779=42, srv962542564=97, srv2001602751=44, srv406621625=64, srv1652899992=22, srv1818552263=34, srv219693326=51, srv621635165=70, srv1726679887=27, srv626492456=72, srv920901843=91, srv1220757353=6, srv164747465=21, srv198002107=43, srv911022551=90, srv185040309=38, srv829973161=84, srv597266931=69, srv550992595=67, srv1416700660=14, srv1019606751=1, srv1485996259=16, srv205255585=47, srv348340156=59, srv1853418669=39, srv1779848597=32, srv782473235=83, srv2134166520=49, srv930587663=92, srv1687117649=25, srv2030316924=46, srv1672314092=23, srv379992304=63, srv549922518=66, srv62706972=74, srv694482373=78, srv1535448962=19, srv975813180=98, srv1731612342=28, srv877267438=88, srv1529394336=18, srv2111204854=48, srv1280533394=11, srv2026405032=45, srv1754991461=30} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:20,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:20,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:20,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:20,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:20,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:20,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-20 00:58:20,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-20 00:58:20,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-20 00:58:20,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-20 00:58:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-20 00:58:20,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-20 00:58:20,718 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1060607277283735E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,719 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv348340156:82 , srv1687117649:83 , srv1829919426:83 , srv2134166520:83 , srv334625049:83 , srv1222416459:84 , srv123930359:84 , srv1485996259:84 , srv1521227373:84 , srv1529394336:84 , srv1535448962:84 , srv1591553004:84 , srv164747465:84 , srv1706273526:84 , srv1726679887:84 , srv1740207560:84 , srv1754991461:84 , srv1779848597:84 , srv1829170310:84 , srv1847267499:84 , srv185040309:84 , srv2001602751:84 , srv2026405032:84 , srv2030316924:84 , srv2136917873:84 , srv270652197:84 , srv294333900:84 , srv32379879:84 , srv326089414:84 , srv379992304:84 , srv597266931:84 , srv625985471:84 , srv626537085:84 , srv62706972:84 , srv693570846:84 , srv724863097:84 , srv911022551:84 , srv937112699:84 , srv946652928:84 , srv1016460172:85 , srv1019606751:85 , srv1186689076:85 , srv1220757353:85 , srv1224136500:85 , srv1239163918:85 , srv1280533394:85 , srv1416700660:85 , srv1431851029:85 , srv1652899992:85 , srv1672314092:85 , srv1677797902:85 , srv1731612342:85 , srv1785313335:85 , srv1818552263:85 , srv1853418669:85 , srv1880039617:85 , srv1958398779:85 , srv205255585:85 , srv2111204854:85 , srv219693326:85 , srv244051217:85 , srv373013607:85 , srv373106243:85 , srv375656478:85 , srv406621625:85 , srv549922518:85 , srv550992595:85 , srv626492456:85 , srv67742609:85 , srv694482373:85 , srv752480339:85 , srv777089497:85 , srv868987788:85 , srv869933052:85 , srv877267438:85 , srv920901843:85 , srv930902737:85 , srv947222909:85 , srv962542564:85 , srv991763763:85 , srv10214305:86 , srv1169317714:86 , srv1181103591:86 , srv1291188301:86 , srv1392911281:86 , srv1758688337:86 , srv1929520444:86 , srv198002107:86 , srv305894910:86 , srv497026709:86 , srv587867855:86 , srv669073724:86 , srv733315781:86 , srv782473235:86 , srv829973161:86 , srv852353570:86 , srv930587663:86 , srv975813180:86 , srv890365972:87 , srv621635165:610 }
2023-03-20 00:58:20,740 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=520 (was 520), ProcessCount=166 (was 166), AvailableMemoryMB=4997 (was 5471)
2023-03-20 00:58:20,746 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=520, ProcessCount=166, AvailableMemoryMB=4997
2023-03-20 00:58:20,747 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-20 00:58:20,747 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-20 00:58:20,757 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100694100:84 , srv1088298634:84 , srv1117507177:84 , srv1237211185:84 , srv1244158994:84 , srv1310008419:84 , srv1421811206:84 , srv1699033201:84 , srv1788241334:84 , srv1817129377:84 , srv1828282559:84 , srv1904330114:84 , srv2009413343:84 , srv2112896886:84 , srv233218345:84 , srv27610717:84 , srv276755692:84 , srv290945074:84 , srv385845913:84 , srv413350155:84 , srv591693371:84 , srv633314724:84 , srv757580963:84 , srv784041068:84 , srv812454020:84 , srv876904280:84 , srv908835331:84 , srv946193811:84 , srv971017581:84 , srv1919341137:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-20 00:58:20,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 00:58:20,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv233218345=15, srv1421811206=6, srv100694100=0, srv1088298634=1, srv385845913=19, srv1699033201=7, srv2009413343=13, srv591693371=21, srv757580963=23, srv784041068=24, srv633314724=22, srv1117507177=2, srv290945074=18, srv1310008419=5, srv1237211185=3, srv876904280=26, srv413350155=20, srv2112896886=14, srv1919341137=12, srv908835331=27, srv1244158994=4, srv1788241334=8, srv1817129377=9, srv1828282559=10, srv1904330114=11, srv946193811=28, srv971017581=29, srv27610717=16, srv812454020=25, srv276755692=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 00:58:20,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 00:58:20,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 00:58:20,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 00:58:20,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 00:58:20,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-20 00:58:20,797 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1502799435732918, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 00:58:20,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912666570459386, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1502799435732918, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-20 01:00:20,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10947448 different iterations.  Found a solution that moves 1830 regions; Going from a computed imbalance of 0.9912666570459386 to a new imbalance of 4.271311292566818E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6777777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 01:00:20,782 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv100694100:90 , srv1088298634:90 , srv1117507177:90 , srv1237211185:90 , srv1244158994:90 , srv1310008419:90 , srv1421811206:90 , srv1699033201:90 , srv1788241334:90 , srv1817129377:90 , srv1828282559:90 , srv1904330114:90 , srv1919341137:90 , srv2009413343:90 , srv2112896886:90 , srv233218345:90 , srv27610717:90 , srv276755692:90 , srv290945074:90 , srv385845913:90 , srv413350155:90 , srv591693371:90 , srv633314724:90 , srv757580963:90 , srv784041068:90 , srv812454020:90 , srv876904280:90 , srv908835331:90 , srv946193811:90 , srv971017581:90 }
2023-03-20 01:00:20,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-20 01:00:20,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv233218345=15, srv1421811206=6, srv100694100=0, srv1088298634=1, srv385845913=19, srv1699033201=7, srv2009413343=13, srv591693371=21, srv757580963=23, srv784041068=24, srv633314724=22, srv1117507177=2, srv290945074=18, srv1310008419=5, srv1237211185=3, srv876904280=26, srv413350155=20, srv2112896886=14, srv1919341137=12, srv908835331=27, srv1244158994=4, srv1788241334=8, srv1817129377=9, srv1828282559=10, srv1904330114=11, srv946193811=28, srv971017581=29, srv27610717=16, srv812454020=25, srv276755692=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-20 01:00:20,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-20 01:00:20,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-20 01:00:20,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-20 01:00:20,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-20 01:00:20,798 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv100694100:90 , srv1088298634:90 , srv1117507177:90 , srv1237211185:90 , srv1244158994:90 , srv1310008419:90 , srv1421811206:90 , srv1699033201:90 , srv1788241334:90 , srv1817129377:90 , srv1828282559:90 , srv1904330114:90 , srv1919341137:90 , srv2009413343:90 , srv2112896886:90 , srv233218345:90 , srv27610717:90 , srv276755692:90 , srv290945074:90 , srv385845913:90 , srv413350155:90 , srv591693371:90 , srv633314724:90 , srv757580963:90 , srv784041068:90 , srv812454020:90 , srv876904280:90 , srv908835331:90 , srv946193811:90 , srv971017581:90 }
2023-03-20 01:00:20,808 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 520), ProcessCount=163 (was 166), AvailableMemoryMB=5832 (was 4997) - AvailableMemoryMB LEAK? -