2023-08-16 02:53:02,517 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-16 02:53:02,766 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-16 02:53:02,856 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-16 02:53:02,911 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 02:53:02,916 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 02:53:02,928 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=174, AvailableMemoryMB=3142
2023-08-16 02:53:02,930 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 02:53:02,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-16 02:53:03,030 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015095255:84 , srv1034498873:84 , srv1048331117:84 , srv1051977763:84 , srv1077105623:84 , srv1085779256:84 , srv1094502593:84 , srv1186736109:84 , srv1196498221:84 , srv1204228112:84 , srv1233043371:84 , srv1247338276:84 , srv1265324286:84 , srv1277553747:84 , srv1299325097:84 , srv1321367848:84 , srv132914234:84 , srv1329886795:84 , srv1353064899:84 , srv1353295906:84 , srv1364505642:84 , srv1432972016:84 , srv1462481694:84 , srv149793493:84 , srv1498407686:84 , srv1500091587:84 , srv1530071410:84 , srv1561480153:84 , srv1562685033:84 , srv1580325084:84 , srv1590843858:84 , srv1595055416:84 , srv1608078537:84 , srv1647151117:84 , srv1649538600:84 , srv1658501176:84 , srv167218538:84 , srv1699427843:84 , srv1719255917:84 , srv1733665380:84 , srv1734212893:84 , srv1736692155:84 , srv1747175313:84 , srv1749721364:84 , srv1761483606:84 , srv1773619448:84 , srv1804354154:84 , srv1824620064:84 , srv1839278792:84 , srv1892714687:84 , srv1908213886:84 , srv1924255626:84 , srv2008319690:84 , srv2030763529:84 , srv2071276930:84 , srv2126473422:84 , srv2143627559:84 , srv236592151:84 , srv248822766:84 , srv283249702:84 , srv30011281:84 , srv301818088:84 , srv311225352:84 , srv312164892:84 , srv318659899:84 , srv341724248:84 , srv363589692:84 , srv366829700:84 , srv37733779:84 , srv393993180:84 , srv405169780:84 , srv45452247:84 , srv496144562:84 , srv514083814:84 , srv517650061:84 , srv543051893:84 , srv546647259:84 , srv555831316:84 , srv618949268:84 , srv62598729:84 , srv669666271:84 , srv673696026:84 , srv674048006:84 , srv680450122:84 , srv695899448:84 , srv715407910:84 , srv719711992:84 , srv749299089:84 , srv753507777:84 , srv756046345:84 , srv784172500:84 , srv801915246:84 , srv872160184:84 , srv874267410:84 , srv874694056:84 , srv876396051:84 , srv884209858:84 , srv9263395:84 , srv960997200:84 , srv933650984:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-16 02:53:03,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:03,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1077105623=4, srv248822766=58, srv680450122=83, srv318659899=64, srv1034498873=1, srv1773619448=45, srv1649538600=34, srv301818088=61, srv236592151=57, srv1749721364=43, srv393993180=69, srv669666271=80, srv695899448=84, srv1736692155=41, srv1747175313=42, srv1364505642=20, srv341724248=65, srv1908213886=50, srv933650984=98, srv2030763529=53, srv2143627559=56, srv1500091587=25, srv2126473422=55, srv1462481694=22, srv1015095255=0, srv363589692=66, srv715407910=85, srv784172500=90, srv673696026=81, srv2008319690=52, srv874267410=93, srv311225352=62, srv756046345=89, srv1196498221=8, srv1299325097=14, srv1924255626=51, srv543051893=75, srv719711992=86, srv876396051=95, srv167218538=36, srv149793493=23, srv1595055416=31, srv1321367848=15, srv30011281=60, srv45452247=71, srv312164892=63, srv872160184=92, srv874694056=94, srv405169780=70, srv555831316=77, srv1265324286=12, srv960997200=99, srv1562685033=28, srv37733779=68, srv1608078537=32, srv1734212893=40, srv1204228112=9, srv1498407686=24, srv1733665380=39, srv2071276930=54, srv801915246=91, srv9263395=97, srv546647259=76, srv1233043371=10, srv1839278792=48, srv1590843858=30, srv1761483606=44, srv618949268=78, srv1247338276=11, srv1277553747=13, srv1530071410=26, srv1353064899=18, srv1186736109=7, srv1561480153=27, srv1094502593=6, srv496144562=72, srv1647151117=33, srv1892714687=49, srv1824620064=47, srv1085779256=5, srv1719255917=38, srv1699427843=37, srv749299089=87, srv753507777=88, srv132914234=16, srv1048331117=2, srv514083814=73, srv674048006=82, srv1804354154=46, srv884209858=96, srv1658501176=35, srv517650061=74, srv1580325084=29, srv366829700=67, srv1329886795=17, srv1353295906=19, srv1051977763=3, srv1432972016=21, srv62598729=79, srv283249702=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:03,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:03,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:03,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:03,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:03,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 02:53:03,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 02:53:03,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 02:53:03,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 02:53:03,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 02:53:03,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 02:53:03,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 02:53:03,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 02:53:03,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 02:53:03,249 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:03,250 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 02:53:03,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 02:53:03,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 329 ms to try 5000 different iterations.  Found a solution that moves 4717 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1777066252252676. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03986477954861794, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05491141728818408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5241111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03986477954861794, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.173, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:03,482 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv680450122:80 , srv749299089:80 , srv669666271:81 , srv1500091587:82 , srv1595055416:82 , srv1647151117:82 , srv517650061:82 , srv1196498221:83 , srv1204228112:83 , srv167218538:83 , srv2008319690:83 , srv312164892:83 , srv366829700:83 , srv9263395:83 , srv1094502593:84 , srv1265324286:84 , srv132914234:84 , srv1580325084:84 , srv1590843858:84 , srv2126473422:84 , srv37733779:84 , srv876396051:84 , srv1015095255:85 , srv1034498873:85 , srv1085779256:85 , srv1186736109:85 , srv1321367848:85 , srv1329886795:85 , srv1364505642:85 , srv1658501176:85 , srv1699427843:85 , srv1804354154:85 , srv2030763529:85 , srv283249702:85 , srv341724248:85 , srv405169780:85 , srv45452247:85 , srv673696026:85 , srv756046345:85 , srv784172500:85 , srv872160184:85 , srv1277553747:86 , srv1299325097:86 , srv1432972016:86 , srv1462481694:86 , srv1530071410:86 , srv1561480153:86 , srv1562685033:86 , srv1773619448:86 , srv1824620064:86 , srv1839278792:86 , srv1908213886:86 , srv1924255626:86 , srv546647259:86 , srv695899448:86 , srv715407910:86 , srv960997200:86 , srv1353295906:87 , srv1498407686:87 , srv1608078537:87 , srv1747175313:87 , srv30011281:87 , srv311225352:87 , srv363589692:87 , srv543051893:87 , srv618949268:87 , srv719711992:87 , srv1051977763:88 , srv1077105623:88 , srv1353064899:88 , srv1649538600:88 , srv1719255917:88 , srv2071276930:88 , srv248822766:88 , srv318659899:88 , srv753507777:88 , srv874267410:88 , srv1247338276:89 , srv149793493:89 , srv1733665380:89 , srv1749721364:89 , srv1892714687:89 , srv2143627559:89 , srv393993180:89 , srv884209858:89 , srv1233043371:90 , srv1734212893:90 , srv1736692155:90 , srv301818088:90 , srv514083814:90 , srv555831316:90 , srv801915246:90 , srv62598729:91 , srv236592151:92 , srv874694056:92 , srv1048331117:93 , srv1761483606:93 , srv674048006:93 , srv496144562:97 , srv933650984:444 }
2023-08-16 02:53:03,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:03,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1077105623=4, srv248822766=58, srv680450122=83, srv318659899=64, srv1034498873=1, srv1773619448=45, srv1649538600=34, srv301818088=61, srv236592151=57, srv1749721364=43, srv393993180=69, srv669666271=80, srv695899448=84, srv1736692155=41, srv1747175313=42, srv1364505642=20, srv341724248=65, srv1908213886=50, srv933650984=98, srv2030763529=53, srv2143627559=56, srv1500091587=25, srv2126473422=55, srv1462481694=22, srv1015095255=0, srv363589692=66, srv715407910=85, srv784172500=90, srv673696026=81, srv2008319690=52, srv874267410=93, srv311225352=62, srv756046345=89, srv1196498221=8, srv1299325097=14, srv1924255626=51, srv543051893=75, srv719711992=86, srv876396051=95, srv167218538=36, srv149793493=23, srv1595055416=31, srv1321367848=15, srv30011281=60, srv45452247=71, srv312164892=63, srv872160184=92, srv874694056=94, srv405169780=70, srv555831316=77, srv1265324286=12, srv960997200=99, srv1562685033=28, srv37733779=68, srv1608078537=32, srv1734212893=40, srv1204228112=9, srv1498407686=24, srv1733665380=39, srv2071276930=54, srv801915246=91, srv9263395=97, srv546647259=76, srv1233043371=10, srv1839278792=48, srv1590843858=30, srv1761483606=44, srv618949268=78, srv1247338276=11, srv1277553747=13, srv1530071410=26, srv1353064899=18, srv1186736109=7, srv1561480153=27, srv1094502593=6, srv496144562=72, srv1647151117=33, srv1892714687=49, srv1824620064=47, srv1085779256=5, srv1719255917=38, srv1699427843=37, srv749299089=87, srv753507777=88, srv132914234=16, srv1048331117=2, srv514083814=73, srv674048006=82, srv1804354154=46, srv884209858=96, srv1658501176=35, srv517650061=74, srv1580325084=29, srv366829700=67, srv1329886795=17, srv1353295906=19, srv1051977763=3, srv1432972016=21, srv62598729=79, srv283249702=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:03,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:03,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:03,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:03,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:03,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:03,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 02:53:03,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 02:53:03,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 02:53:03,512 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 02:53:03,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 02:53:03,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 02:53:03,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 02:53:03,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 02:53:03,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 02:53:03,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 02:53:03,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 02:53:03,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 02:53:03,666 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03986477954861794, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05491141728818408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03986477954861794, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.173, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:03,666 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 02:53:03,666 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17767359608532163, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03986477954861794, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05491141728818408, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03986477954861794, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.173, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 02:53:03,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 314 ms to try 5000 different iterations.  Found a solution that moves 2603 regions; Going from a computed imbalance of 0.17767359608532163 to a new imbalance of 0.013350057106159566. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031337559712098124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2892222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:03,859 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv496144562:76 , srv1048331117:77 , srv874694056:77 , srv1761483606:80 , srv674048006:80 , srv801915246:80 , srv1247338276:81 , srv236592151:81 , srv301818088:81 , srv555831316:81 , srv62598729:81 , srv874267410:81 , srv1051977763:82 , srv1233043371:82 , srv1353295906:82 , srv149793493:82 , srv1733665380:82 , srv1734212893:82 , srv1736692155:82 , srv363589692:82 , srv1749721364:83 , srv1892714687:83 , srv1277553747:84 , srv1649538600:84 , srv1719255917:84 , srv1747175313:84 , srv2143627559:84 , srv248822766:84 , srv753507777:84 , srv1077105623:85 , srv1498407686:85 , srv1608078537:85 , srv1924255626:85 , srv2030763529:85 , srv393993180:85 , srv514083814:85 , srv546647259:85 , srv695899448:85 , srv719711992:85 , srv872160184:85 , srv884209858:85 , srv1034498873:86 , srv1186736109:86 , srv1321367848:86 , srv1353064899:86 , srv1364505642:86 , srv1462481694:86 , srv1562685033:86 , srv1580325084:86 , srv1658501176:86 , srv1699427843:86 , srv2071276930:86 , srv2126473422:86 , srv37733779:86 , srv543051893:86 , srv618949268:86 , srv1085779256:87 , srv1094502593:87 , srv1530071410:87 , srv1561480153:87 , srv1804354154:87 , srv1824620064:87 , srv1908213886:87 , srv283249702:87 , srv311225352:87 , srv318659899:87 , srv341724248:87 , srv405169780:87 , srv45452247:87 , srv715407910:87 , srv756046345:87 , srv876396051:87 , srv960997200:87 , srv1299325097:88 , srv1329886795:88 , srv1432972016:88 , srv1590843858:88 , srv1773619448:88 , srv1839278792:88 , srv30011281:88 , srv366829700:88 , srv673696026:88 , srv9263395:88 , srv1015095255:89 , srv1196498221:89 , srv1204228112:89 , srv167218538:89 , srv2008319690:89 , srv312164892:89 , srv517650061:89 , srv784172500:89 , srv1265324286:90 , srv132914234:90 , srv1500091587:90 , srv1595055416:90 , srv1647151117:90 , srv669666271:91 , srv680450122:92 , srv749299089:93 , srv933650984:527 }
2023-08-16 02:53:03,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:03,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1077105623=4, srv248822766=58, srv680450122=83, srv318659899=64, srv1034498873=1, srv1773619448=45, srv1649538600=34, srv301818088=61, srv236592151=57, srv1749721364=43, srv393993180=69, srv669666271=80, srv695899448=84, srv1736692155=41, srv1747175313=42, srv1364505642=20, srv341724248=65, srv1908213886=50, srv933650984=98, srv2030763529=53, srv2143627559=56, srv1500091587=25, srv2126473422=55, srv1462481694=22, srv1015095255=0, srv363589692=66, srv715407910=85, srv784172500=90, srv673696026=81, srv2008319690=52, srv874267410=93, srv311225352=62, srv756046345=89, srv1196498221=8, srv1299325097=14, srv1924255626=51, srv543051893=75, srv719711992=86, srv876396051=95, srv167218538=36, srv149793493=23, srv1595055416=31, srv1321367848=15, srv30011281=60, srv45452247=71, srv312164892=63, srv872160184=92, srv874694056=94, srv405169780=70, srv555831316=77, srv1265324286=12, srv960997200=99, srv1562685033=28, srv37733779=68, srv1608078537=32, srv1734212893=40, srv1204228112=9, srv1498407686=24, srv1733665380=39, srv2071276930=54, srv801915246=91, srv9263395=97, srv546647259=76, srv1233043371=10, srv1839278792=48, srv1590843858=30, srv1761483606=44, srv618949268=78, srv1247338276=11, srv1277553747=13, srv1530071410=26, srv1353064899=18, srv1186736109=7, srv1561480153=27, srv1094502593=6, srv496144562=72, srv1647151117=33, srv1892714687=49, srv1824620064=47, srv1085779256=5, srv1719255917=38, srv1699427843=37, srv749299089=87, srv753507777=88, srv132914234=16, srv1048331117=2, srv514083814=73, srv674048006=82, srv1804354154=46, srv884209858=96, srv1658501176=35, srv517650061=74, srv1580325084=29, srv366829700=67, srv1329886795=17, srv1353295906=19, srv1051977763=3, srv1432972016=21, srv62598729=79, srv283249702=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:03,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:03,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:03,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:03,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:03,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 02:53:03,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 02:53:03,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 02:53:03,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 02:53:03,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 02:53:03,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 02:53:03,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 02:53:03,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031337559712098124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:03,972 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 02:53:03,972 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013331830510594727, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02216378399242671); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031337559712098124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02216378399242671); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 02:53:04,143 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 243 ms to try 5000 different iterations.  Found a solution that moves 930 regions; Going from a computed imbalance of 0.013331830510594727 to a new imbalance of 0.0011465846964819144. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006763236226828305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00936838056620359); MoveCostFunction : (multiplier=7.0, imbalance=0.10333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006763236226828305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:04,145 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv514083814:82 , srv1015095255:83 , srv1299325097:83 , srv132914234:83 , srv1353064899:83 , srv1773619448:83 , srv1839278792:83 , srv2071276930:83 , srv30011281:83 , srv311225352:83 , srv318659899:83 , srv784172500:83 , srv1077105623:84 , srv1265324286:84 , srv1329886795:84 , srv1432972016:84 , srv1500091587:84 , srv1530071410:84 , srv1761483606:84 , srv2143627559:84 , srv236592151:84 , srv393993180:84 , srv618949268:84 , srv673696026:84 , srv674048006:84 , srv680450122:84 , srv749299089:84 , srv884209858:84 , srv1204228112:85 , srv1233043371:85 , srv1462481694:85 , srv1561480153:85 , srv1590843858:85 , srv1595055416:85 , srv1647151117:85 , srv167218538:85 , srv1719255917:85 , srv1734212893:85 , srv1749721364:85 , srv1804354154:85 , srv1824620064:85 , srv1908213886:85 , srv2008319690:85 , srv248822766:85 , srv312164892:85 , srv341724248:85 , srv405169780:85 , srv496144562:85 , srv543051893:85 , srv669666271:85 , srv715407910:85 , srv753507777:85 , srv756046345:85 , srv960997200:85 , srv1034498873:86 , srv1085779256:86 , srv1094502593:86 , srv1196498221:86 , srv1498407686:86 , srv1562685033:86 , srv1608078537:86 , srv1649538600:86 , srv1699427843:86 , srv1736692155:86 , srv1747175313:86 , srv1892714687:86 , srv1924255626:86 , srv283249702:86 , srv45452247:86 , srv546647259:86 , srv62598729:86 , srv695899448:86 , srv719711992:86 , srv9263395:86 , srv1048331117:87 , srv1051977763:87 , srv1186736109:87 , srv1277553747:87 , srv1321367848:87 , srv1364505642:87 , srv1580325084:87 , srv1658501176:87 , srv1733665380:87 , srv2030763529:87 , srv301818088:87 , srv517650061:87 , srv555831316:87 , srv801915246:87 , srv872160184:87 , srv876396051:87 , srv1247338276:88 , srv1353295906:88 , srv149793493:88 , srv2126473422:88 , srv363589692:88 , srv366829700:88 , srv37733779:88 , srv874267410:88 , srv874694056:88 , srv933650984:547 }
2023-08-16 02:53:04,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:04,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1077105623=4, srv248822766=58, srv680450122=83, srv318659899=64, srv1034498873=1, srv1773619448=45, srv1649538600=34, srv301818088=61, srv236592151=57, srv1749721364=43, srv393993180=69, srv669666271=80, srv695899448=84, srv1736692155=41, srv1747175313=42, srv1364505642=20, srv341724248=65, srv1908213886=50, srv933650984=98, srv2030763529=53, srv2143627559=56, srv1500091587=25, srv2126473422=55, srv1462481694=22, srv1015095255=0, srv363589692=66, srv715407910=85, srv784172500=90, srv673696026=81, srv2008319690=52, srv874267410=93, srv311225352=62, srv756046345=89, srv1196498221=8, srv1299325097=14, srv1924255626=51, srv543051893=75, srv719711992=86, srv876396051=95, srv167218538=36, srv149793493=23, srv1595055416=31, srv1321367848=15, srv30011281=60, srv45452247=71, srv312164892=63, srv872160184=92, srv874694056=94, srv405169780=70, srv555831316=77, srv1265324286=12, srv960997200=99, srv1562685033=28, srv37733779=68, srv1608078537=32, srv1734212893=40, srv1204228112=9, srv1498407686=24, srv1733665380=39, srv2071276930=54, srv801915246=91, srv9263395=97, srv546647259=76, srv1233043371=10, srv1839278792=48, srv1590843858=30, srv1761483606=44, srv618949268=78, srv1247338276=11, srv1277553747=13, srv1530071410=26, srv1353064899=18, srv1186736109=7, srv1561480153=27, srv1094502593=6, srv496144562=72, srv1647151117=33, srv1892714687=49, srv1824620064=47, srv1085779256=5, srv1719255917=38, srv1699427843=37, srv749299089=87, srv753507777=88, srv132914234=16, srv1048331117=2, srv514083814=73, srv674048006=82, srv1804354154=46, srv884209858=96, srv1658501176=35, srv517650061=74, srv1580325084=29, srv366829700=67, srv1329886795=17, srv1353295906=19, srv1051977763=3, srv1432972016=21, srv62598729=79, srv283249702=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:04,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:04,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:04,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:04,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 02:53:04,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 02:53:04,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 02:53:04,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 02:53:04,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 02:53:04,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 02:53:04,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 02:53:04,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 02:53:04,254 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006763236226828305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00936838056620359); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006763236226828305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:04,255 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-16 02:53:04,255 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.001140072697298165, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006763236226828305); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00936838056620359); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006763236226828305); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-16 02:53:04,477 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 293 ms to try 5000 different iterations.  Found a solution that moves 295 regions; Going from a computed imbalance of 0.001140072697298165 to a new imbalance of 2.1746406532108197E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.03277777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:04,478 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv149793493:83 , srv366829700:83 , srv37733779:83 , srv517650061:83 , srv1085779256:84 , srv1094502593:84 , srv1186736109:84 , srv1196498221:84 , srv1247338276:84 , srv1265324286:84 , srv1364505642:84 , srv1462481694:84 , srv1498407686:84 , srv1530071410:84 , srv1561480153:84 , srv1590843858:84 , srv1608078537:84 , srv1649538600:84 , srv1658501176:84 , srv1733665380:84 , srv1736692155:84 , srv1892714687:84 , srv1908213886:84 , srv2126473422:84 , srv283249702:84 , srv30011281:84 , srv301818088:84 , srv318659899:84 , srv393993180:84 , srv45452247:84 , srv496144562:84 , srv555831316:84 , srv62598729:84 , srv674048006:84 , srv715407910:84 , srv719711992:84 , srv874694056:84 , srv876396051:84 , srv884209858:84 , srv960997200:84 , srv1015095255:85 , srv1034498873:85 , srv1048331117:85 , srv1051977763:85 , srv1077105623:85 , srv1204228112:85 , srv1233043371:85 , srv1299325097:85 , srv1321367848:85 , srv132914234:85 , srv1329886795:85 , srv1353064899:85 , srv1353295906:85 , srv1432972016:85 , srv1500091587:85 , srv1562685033:85 , srv1580325084:85 , srv1595055416:85 , srv1647151117:85 , srv167218538:85 , srv1699427843:85 , srv1719255917:85 , srv1734212893:85 , srv1747175313:85 , srv1749721364:85 , srv1761483606:85 , srv1773619448:85 , srv1804354154:85 , srv1824620064:85 , srv1839278792:85 , srv1924255626:85 , srv2008319690:85 , srv2030763529:85 , srv2071276930:85 , srv2143627559:85 , srv236592151:85 , srv248822766:85 , srv311225352:85 , srv312164892:85 , srv341724248:85 , srv363589692:85 , srv405169780:85 , srv514083814:85 , srv543051893:85 , srv546647259:85 , srv618949268:85 , srv669666271:85 , srv695899448:85 , srv749299089:85 , srv756046345:85 , srv784172500:85 , srv872160184:85 , srv874267410:85 , srv1277553747:86 , srv673696026:86 , srv680450122:86 , srv753507777:86 , srv801915246:86 , srv9263395:86 , srv933650984:623 }
2023-08-16 02:53:04,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:04,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1077105623=4, srv248822766=58, srv680450122=83, srv318659899=64, srv1034498873=1, srv1773619448=45, srv1649538600=34, srv301818088=61, srv236592151=57, srv1749721364=43, srv393993180=69, srv669666271=80, srv695899448=84, srv1736692155=41, srv1747175313=42, srv1364505642=20, srv341724248=65, srv1908213886=50, srv933650984=98, srv2030763529=53, srv2143627559=56, srv1500091587=25, srv2126473422=55, srv1462481694=22, srv1015095255=0, srv363589692=66, srv715407910=85, srv784172500=90, srv673696026=81, srv2008319690=52, srv874267410=93, srv311225352=62, srv756046345=89, srv1196498221=8, srv1299325097=14, srv1924255626=51, srv543051893=75, srv719711992=86, srv876396051=95, srv167218538=36, srv149793493=23, srv1595055416=31, srv1321367848=15, srv30011281=60, srv45452247=71, srv312164892=63, srv872160184=92, srv874694056=94, srv405169780=70, srv555831316=77, srv1265324286=12, srv960997200=99, srv1562685033=28, srv37733779=68, srv1608078537=32, srv1734212893=40, srv1204228112=9, srv1498407686=24, srv1733665380=39, srv2071276930=54, srv801915246=91, srv9263395=97, srv546647259=76, srv1233043371=10, srv1839278792=48, srv1590843858=30, srv1761483606=44, srv618949268=78, srv1247338276=11, srv1277553747=13, srv1530071410=26, srv1353064899=18, srv1186736109=7, srv1561480153=27, srv1094502593=6, srv496144562=72, srv1647151117=33, srv1892714687=49, srv1824620064=47, srv1085779256=5, srv1719255917=38, srv1699427843=37, srv749299089=87, srv753507777=88, srv132914234=16, srv1048331117=2, srv514083814=73, srv674048006=82, srv1804354154=46, srv884209858=96, srv1658501176=35, srv517650061=74, srv1580325084=29, srv366829700=67, srv1329886795=17, srv1353295906=19, srv1051977763=3, srv1432972016=21, srv62598729=79, srv283249702=59} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:04,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:04,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-16 02:53:04,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-16 02:53:04,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-16 02:53:04,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-16 02:53:04,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-16 02:53:04,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-16 02:53:04,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-16 02:53:04,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-16 02:53:04,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.9680772382424242E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:04,574 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv149793493:83 , srv366829700:83 , srv37733779:83 , srv517650061:83 , srv1085779256:84 , srv1094502593:84 , srv1186736109:84 , srv1196498221:84 , srv1247338276:84 , srv1265324286:84 , srv1364505642:84 , srv1462481694:84 , srv1498407686:84 , srv1530071410:84 , srv1561480153:84 , srv1590843858:84 , srv1608078537:84 , srv1649538600:84 , srv1658501176:84 , srv1733665380:84 , srv1736692155:84 , srv1892714687:84 , srv1908213886:84 , srv2126473422:84 , srv283249702:84 , srv30011281:84 , srv301818088:84 , srv318659899:84 , srv393993180:84 , srv45452247:84 , srv496144562:84 , srv555831316:84 , srv62598729:84 , srv674048006:84 , srv715407910:84 , srv719711992:84 , srv874694056:84 , srv876396051:84 , srv884209858:84 , srv960997200:84 , srv1015095255:85 , srv1034498873:85 , srv1048331117:85 , srv1051977763:85 , srv1077105623:85 , srv1204228112:85 , srv1233043371:85 , srv1299325097:85 , srv1321367848:85 , srv132914234:85 , srv1329886795:85 , srv1353064899:85 , srv1353295906:85 , srv1432972016:85 , srv1500091587:85 , srv1562685033:85 , srv1580325084:85 , srv1595055416:85 , srv1647151117:85 , srv167218538:85 , srv1699427843:85 , srv1719255917:85 , srv1734212893:85 , srv1747175313:85 , srv1749721364:85 , srv1761483606:85 , srv1773619448:85 , srv1804354154:85 , srv1824620064:85 , srv1839278792:85 , srv1924255626:85 , srv2008319690:85 , srv2030763529:85 , srv2071276930:85 , srv2143627559:85 , srv236592151:85 , srv248822766:85 , srv311225352:85 , srv312164892:85 , srv341724248:85 , srv363589692:85 , srv405169780:85 , srv514083814:85 , srv543051893:85 , srv546647259:85 , srv618949268:85 , srv669666271:85 , srv695899448:85 , srv749299089:85 , srv756046345:85 , srv784172500:85 , srv872160184:85 , srv874267410:85 , srv1277553747:86 , srv673696026:86 , srv680450122:86 , srv753507777:86 , srv801915246:86 , srv9263395:86 , srv933650984:623 }
2023-08-16 02:53:04,600 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=337 (was 340), ProcessCount=174 (was 174), AvailableMemoryMB=2675 (was 3142)
2023-08-16 02:53:04,607 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=174, AvailableMemoryMB=2675
2023-08-16 02:53:04,607 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-16 02:53:04,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-16 02:53:04,618 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1043802642:84 , srv1160920114:84 , srv1251525424:84 , srv1313301641:84 , srv1318955634:84 , srv1466446225:84 , srv1520008227:84 , srv1525651153:84 , srv1540247637:84 , srv1670883939:84 , srv1712002951:84 , srv171351328:84 , srv1731213287:84 , srv1903313263:84 , srv1989964727:84 , srv2083311466:84 , srv258393335:84 , srv273085101:84 , srv358108700:84 , srv399524127:84 , srv450148401:84 , srv487756450:84 , srv521633319:84 , srv54278279:84 , srv570985240:84 , srv592259747:84 , srv720776364:84 , srv772738053:84 , srv846302846:84 , srv1153262355:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-16 02:53:04,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:53:04,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1466446225=6, srv846302846=29, srv1160920114=2, srv1989964727=15, srv592259747=26, srv570985240=25, srv1903313263=14, srv399524127=20, srv2083311466=16, srv1318955634=5, srv1540247637=9, srv450148401=21, srv521633319=23, srv772738053=28, srv1043802642=0, srv358108700=19, srv54278279=24, srv1670883939=10, srv720776364=27, srv171351328=12, srv273085101=18, srv487756450=22, srv1520008227=7, srv1153262355=1, srv1525651153=8, srv1313301641=4, srv1731213287=13, srv1251525424=3, srv1712002951=11, srv258393335=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:53:04,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:53:04,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:53:04,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:53:04,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 02:53:04,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1558496857320266, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:53:04,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991268412053506, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1558496857320266, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-16 02:55:04,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9954078 different iterations.  Found a solution that moves 1835 regions; Going from a computed imbalance of 0.991268412053506 to a new imbalance of 4.2829815420000605E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6796296296296296, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:55:04,645 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1043802642:90 , srv1153262355:90 , srv1160920114:90 , srv1251525424:90 , srv1313301641:90 , srv1318955634:90 , srv1466446225:90 , srv1520008227:90 , srv1525651153:90 , srv1540247637:90 , srv1670883939:90 , srv1712002951:90 , srv171351328:90 , srv1731213287:90 , srv1903313263:90 , srv1989964727:90 , srv2083311466:90 , srv258393335:90 , srv273085101:90 , srv358108700:90 , srv399524127:90 , srv450148401:90 , srv487756450:90 , srv521633319:90 , srv54278279:90 , srv570985240:90 , srv592259747:90 , srv720776364:90 , srv772738053:90 , srv846302846:90 }
2023-08-16 02:55:04,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-16 02:55:04,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1466446225=6, srv846302846=29, srv1160920114=2, srv1989964727=15, srv592259747=26, srv570985240=25, srv1903313263=14, srv399524127=20, srv2083311466=16, srv1318955634=5, srv1540247637=9, srv450148401=21, srv521633319=23, srv772738053=28, srv1043802642=0, srv358108700=19, srv54278279=24, srv1670883939=10, srv720776364=27, srv171351328=12, srv273085101=18, srv487756450=22, srv1520008227=7, srv1153262355=1, srv1525651153=8, srv1313301641=4, srv1731213287=13, srv1251525424=3, srv1712002951=11, srv258393335=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-16 02:55:04,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-16 02:55:04,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-16 02:55:04,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-16 02:55:04,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-16 02:55:04,659 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-16 02:55:04,659 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1043802642:90 , srv1153262355:90 , srv1160920114:90 , srv1251525424:90 , srv1313301641:90 , srv1318955634:90 , srv1466446225:90 , srv1520008227:90 , srv1525651153:90 , srv1540247637:90 , srv1670883939:90 , srv1712002951:90 , srv171351328:90 , srv1731213287:90 , srv1903313263:90 , srv1989964727:90 , srv2083311466:90 , srv258393335:90 , srv273085101:90 , srv358108700:90 , srv399524127:90 , srv450148401:90 , srv487756450:90 , srv521633319:90 , srv54278279:90 , srv570985240:90 , srv592259747:90 , srv720776364:90 , srv772738053:90 , srv846302846:90 }
2023-08-16 02:55:04,670 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 337), ProcessCount=173 (was 174), AvailableMemoryMB=2506 (was 2675)