2023-07-11 20:34:06,859 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-11 20:34:07,286 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-11 20:34:07,443 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-11 20:34:07,536 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-11 20:34:07,547 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-11 20:34:07,565 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=373, ProcessCount=166, AvailableMemoryMB=9238
2023-07-11 20:34:07,568 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-11 20:34:07,569 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-11 20:34:07,808 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1022160506:84 , srv1035486473:84 , srv1039428461:84 , srv1090236618:84 , srv1173917895:84 , srv1179978579:84 , srv1196313829:84 , srv1225462505:84 , srv1228154989:84 , srv1229478515:84 , srv1251312002:84 , srv1297962039:84 , srv1348778385:84 , srv1389714021:84 , srv1422543242:84 , srv1423940427:84 , srv144427312:84 , srv1461003143:84 , srv1462867295:84 , srv1464193746:84 , srv1534824192:84 , srv1541312122:84 , srv1542800665:84 , srv1600313655:84 , srv1630715226:84 , srv1644734683:84 , srv1644997594:84 , srv1706758030:84 , srv1712631455:84 , srv1740862808:84 , srv1769573138:84 , srv1774644222:84 , srv178597972:84 , srv1793002439:84 , srv1796657214:84 , srv1811369685:84 , srv1821747421:84 , srv1838425314:84 , srv1846026393:84 , srv1853564796:84 , srv186425014:84 , srv1865596283:84 , srv1873124449:84 , srv1911713832:84 , srv1963907863:84 , srv1991753655:84 , srv1992646250:84 , srv1997142553:84 , srv2011493255:84 , srv2013506564:84 , srv2038547076:84 , srv2096161618:84 , srv209828816:84 , srv2098898801:84 , srv2116376573:84 , srv2128455015:84 , srv256219399:84 , srv263928717:84 , srv269806292:84 , srv275116991:84 , srv303192302:84 , srv327462984:84 , srv365406819:84 , srv384192012:84 , srv415910992:84 , srv510147641:84 , srv51031370:84 , srv530840180:84 , srv537932445:84 , srv557009583:84 , srv584670424:84 , srv590851086:84 , srv598368707:84 , srv639165207:84 , srv66079082:84 , srv664463959:84 , srv669175080:84 , srv699618549:84 , srv700609737:84 , srv722346024:84 , srv724798958:84 , srv740586477:84 , srv764509472:84 , srv779335656:84 , srv7826877:84 , srv784358768:84 , srv814878733:84 , srv842804091:84 , srv878086313:84 , srv891762054:84 , srv897253536:84 , srv905737253:84 , srv921539741:84 , srv949117111:84 , srv962460307:84 , srv966123914:84 , srv979837431:84 , srv992886659:84 , srv999856235:84 , srv967405118:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-11 20:34:07,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:07,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:08,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:08,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:08,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:08,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:08,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:08,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:08,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:08,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:08,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:08,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:08,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:08,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:08,348 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:08,348 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-11 20:34:08,349 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-11 20:34:09,246 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1168 ms to try 5000 different iterations.  Found a solution that moves 4716 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17508380501528156. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037292051356834714, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05332912441304635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.524, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037292051356834714, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:09,269 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1196313829:81 , srv1542800665:81 , srv2096161618:82 , srv269806292:82 , srv598368707:82 , srv1228154989:83 , srv1992646250:83 , srv275116991:83 , srv7826877:83 , srv814878733:83 , srv962460307:83 , srv992886659:83 , srv1035486473:84 , srv1229478515:84 , srv1464193746:84 , srv1644734683:84 , srv1740862808:84 , srv1838425314:84 , srv1865596283:84 , srv2128455015:84 , srv256219399:84 , srv415910992:84 , srv724798958:84 , srv999856235:84 , srv1251312002:85 , srv144427312:85 , srv1774644222:85 , srv1796657214:85 , srv1963907863:85 , srv1991753655:85 , srv2098898801:85 , srv263928717:85 , srv537932445:85 , srv639165207:85 , srv699618549:85 , srv700609737:85 , srv740586477:85 , srv764509472:85 , srv1039428461:86 , srv1225462505:86 , srv1423940427:86 , srv1600313655:86 , srv1706758030:86 , srv186425014:86 , srv1997142553:86 , srv2013506564:86 , srv779335656:86 , srv878086313:86 , srv891762054:86 , srv905737253:86 , srv979837431:86 , srv1090236618:87 , srv1179978579:87 , srv1297962039:87 , srv1348778385:87 , srv1462867295:87 , srv1769573138:87 , srv1846026393:87 , srv530840180:87 , srv557009583:87 , srv949117111:87 , srv966123914:87 , srv1461003143:88 , srv1541312122:88 , srv1630715226:88 , srv178597972:88 , srv1793002439:88 , srv1811369685:88 , srv1853564796:88 , srv2038547076:88 , srv2116376573:88 , srv303192302:88 , srv584670424:88 , srv669175080:88 , srv722346024:88 , srv784358768:88 , srv921539741:88 , srv1022160506:89 , srv2011493255:89 , srv327462984:89 , srv51031370:89 , srv590851086:89 , srv66079082:89 , srv664463959:89 , srv842804091:89 , srv1173917895:90 , srv1422543242:90 , srv1644997594:90 , srv209828816:90 , srv510147641:90 , srv1712631455:91 , srv1911713832:91 , srv365406819:91 , srv897253536:91 , srv1534824192:92 , srv1389714021:94 , srv1821747421:94 , srv1873124449:94 , srv384192012:97 , srv967405118:421 }
2023-07-11 20:34:09,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:09,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:09,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:09,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:09,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:09,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:09,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:09,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:09,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:09,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:09,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:09,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:09,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:09,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:09,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:09,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:09,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:09,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:09,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:09,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:09,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:09,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:09,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:09,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:09,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:09,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:09,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:09,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:09,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:09,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:09,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:09,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:09,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:09,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:09,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:09,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:09,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:09,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:09,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:09,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:09,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:09,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:09,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:09,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:09,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:09,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:09,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:09,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:09,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:09,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:09,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:09,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:09,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:09,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:09,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:09,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:09,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:09,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:09,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:09,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:09,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:09,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:09,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:09,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:09,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:09,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037292051356834714, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05332912441304635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037292051356834714, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:09,655 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-11 20:34:09,655 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17505078287748524, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037292051356834714, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05332912441304635, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.037292051356834714, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1705, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-11 20:34:10,193 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 794 ms to try 5000 different iterations.  Found a solution that moves 2590 regions; Going from a computed imbalance of 0.17505078287748524 to a new imbalance of 0.013486111425987556. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019247600718871338); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031369776070758255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019247600718871338); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:10,208 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1389714021:77 , srv1821747421:77 , srv384192012:78 , srv1873124449:79 , srv1534824192:80 , srv1712631455:80 , srv365406819:80 , srv897253536:80 , srv510147641:81 , srv1173917895:82 , srv1644997594:82 , srv1911713832:82 , srv209828816:82 , srv327462984:82 , srv1793002439:83 , srv303192302:83 , srv664463959:83 , srv722346024:83 , srv1179978579:84 , srv1348778385:84 , srv1422543242:84 , srv1461003143:84 , srv1769573138:84 , srv1811369685:84 , srv1846026393:84 , srv186425014:84 , srv1997142553:84 , srv2116376573:84 , srv51031370:84 , srv590851086:84 , srv66079082:84 , srv669175080:84 , srv784358768:84 , srv921539741:84 , srv966123914:84 , srv1039428461:85 , srv1090236618:85 , srv1423940427:85 , srv1462867295:85 , srv1541312122:85 , srv1706758030:85 , srv178597972:85 , srv1853564796:85 , srv2011493255:85 , srv584670424:85 , srv842804091:85 , srv891762054:85 , srv979837431:85 , srv1022160506:86 , srv1630715226:86 , srv1963907863:86 , srv1991753655:86 , srv2013506564:86 , srv2038547076:86 , srv263928717:86 , srv699618549:86 , srv878086313:86 , srv949117111:86 , srv1225462505:87 , srv1251312002:87 , srv1297962039:87 , srv1464193746:87 , srv1774644222:87 , srv2098898801:87 , srv2128455015:87 , srv530840180:87 , srv537932445:87 , srv740586477:87 , srv1035486473:88 , srv144427312:88 , srv1600313655:88 , srv1740862808:88 , srv1796657214:88 , srv1838425314:88 , srv1865596283:88 , srv1992646250:88 , srv275116991:88 , srv415910992:88 , srv779335656:88 , srv7826877:88 , srv905737253:88 , srv962460307:88 , srv992886659:88 , srv1228154989:89 , srv1229478515:89 , srv2096161618:89 , srv256219399:89 , srv557009583:89 , srv700609737:89 , srv724798958:89 , srv999856235:89 , srv1542800665:90 , srv1644734683:90 , srv639165207:90 , srv814878733:90 , srv1196313829:91 , srv764509472:91 , srv598368707:92 , srv269806292:93 , srv967405118:524 }
2023-07-11 20:34:10,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:10,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:10,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:10,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:10,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:10,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:10,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:10,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:10,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:10,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:10,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:10,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:10,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:10,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:10,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:10,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:10,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:10,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:10,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:10,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:10,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:10,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:10,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:10,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:10,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:10,250 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:10,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:10,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:10,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:10,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:10,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:10,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:10,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019247600718871338); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031369776070758255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019247600718871338); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:10,460 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-11 20:34:10,460 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013467975858368297, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019247600718871338); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031369776070758255, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019247600718871338); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-11 20:34:10,878 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 586 ms to try 5000 different iterations.  Found a solution that moves 962 regions; Going from a computed imbalance of 0.013467975858368297 to a new imbalance of 5.578331338107595E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007927845200140842); MoveCostFunction : (multiplier=7.0, imbalance=0.1068888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:10,881 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1022160506:82 , srv269806292:82 , srv384192012:82 , srv764509472:82 , srv1600313655:83 , srv1644734683:83 , srv530840180:83 , srv557009583:83 , srv639165207:83 , srv700609737:83 , srv779335656:83 , srv1297962039:84 , srv1422543242:84 , srv1541312122:84 , srv1630715226:84 , srv1796657214:84 , srv1853564796:84 , srv2011493255:84 , srv256219399:84 , srv415910992:84 , srv598368707:84 , srv724798958:84 , srv814878733:84 , srv842804091:84 , srv905737253:84 , srv949117111:84 , srv999856235:84 , srv1090236618:85 , srv1173917895:85 , srv1196313829:85 , srv1225462505:85 , srv1229478515:85 , srv1348778385:85 , srv144427312:85 , srv1462867295:85 , srv1644997594:85 , srv178597972:85 , srv1793002439:85 , srv1838425314:85 , srv1865596283:85 , srv1911713832:85 , srv2013506564:85 , srv2038547076:85 , srv51031370:85 , srv537932445:85 , srv584670424:85 , srv590851086:85 , srv66079082:85 , srv664463959:85 , srv740586477:85 , srv921539741:85 , srv1035486473:86 , srv1228154989:86 , srv1251312002:86 , srv1461003143:86 , srv1464193746:86 , srv1534824192:86 , srv1542800665:86 , srv1706758030:86 , srv1712631455:86 , srv1740862808:86 , srv1769573138:86 , srv1774644222:86 , srv1811369685:86 , srv1821747421:86 , srv1873124449:86 , srv1963907863:86 , srv1991753655:86 , srv1992646250:86 , srv209828816:86 , srv2098898801:86 , srv2116376573:86 , srv2128455015:86 , srv275116991:86 , srv365406819:86 , srv510147641:86 , srv669175080:86 , srv722346024:86 , srv7826877:86 , srv784358768:86 , srv878086313:86 , srv891762054:86 , srv897253536:86 , srv979837431:86 , srv992886659:86 , srv1039428461:87 , srv1179978579:87 , srv1389714021:87 , srv1423940427:87 , srv1846026393:87 , srv1997142553:87 , srv2096161618:87 , srv263928717:87 , srv303192302:87 , srv327462984:87 , srv699618549:87 , srv962460307:87 , srv966123914:87 , srv186425014:88 , srv967405118:564 }
2023-07-11 20:34:10,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:10,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:10,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:10,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:10,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:10,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:10,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:10,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:10,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:10,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:10,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:10,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:10,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:10,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:10,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:10,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:10,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:10,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:10,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:10,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:10,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:10,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:10,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:10,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:10,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:10,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:10,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:10,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:10,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:11,118 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007927845200140842); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:11,119 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-11 20:34:11,119 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.510970658378918E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007927845200140842); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00375); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-11 20:34:11,527 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 570 ms to try 5000 different iterations.  Found a solution that moves 276 regions; Going from a computed imbalance of 5.510970658378918E-4 to a new imbalance of 6.53600367873526E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.030666666666666665, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:11,528 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1873124449:83 , srv2038547076:83 , srv557009583:83 , srv878086313:83 , srv1035486473:84 , srv1179978579:84 , srv1196313829:84 , srv1225462505:84 , srv1228154989:84 , srv1229478515:84 , srv1251312002:84 , srv1297962039:84 , srv1422543242:84 , srv1423940427:84 , srv144427312:84 , srv1461003143:84 , srv1534824192:84 , srv1630715226:84 , srv1740862808:84 , srv1774644222:84 , srv178597972:84 , srv1811369685:84 , srv1821747421:84 , srv1846026393:84 , srv186425014:84 , srv1911713832:84 , srv2011493255:84 , srv2096161618:84 , srv209828816:84 , srv2098898801:84 , srv2116376573:84 , srv263928717:84 , srv275116991:84 , srv303192302:84 , srv327462984:84 , srv51031370:84 , srv590851086:84 , srv598368707:84 , srv639165207:84 , srv66079082:84 , srv669175080:84 , srv699618549:84 , srv724798958:84 , srv764509472:84 , srv784358768:84 , srv842804091:84 , srv905737253:84 , srv921539741:84 , srv962460307:84 , srv966123914:84 , srv1022160506:85 , srv1039428461:85 , srv1090236618:85 , srv1173917895:85 , srv1389714021:85 , srv1462867295:85 , srv1464193746:85 , srv1541312122:85 , srv1542800665:85 , srv1600313655:85 , srv1644734683:85 , srv1644997594:85 , srv1706758030:85 , srv1712631455:85 , srv1769573138:85 , srv1838425314:85 , srv1853564796:85 , srv1865596283:85 , srv1963907863:85 , srv1992646250:85 , srv1997142553:85 , srv2013506564:85 , srv2128455015:85 , srv256219399:85 , srv365406819:85 , srv384192012:85 , srv510147641:85 , srv530840180:85 , srv537932445:85 , srv584670424:85 , srv664463959:85 , srv700609737:85 , srv722346024:85 , srv740586477:85 , srv779335656:85 , srv7826877:85 , srv814878733:85 , srv891762054:85 , srv897253536:85 , srv949117111:85 , srv979837431:85 , srv992886659:85 , srv999856235:85 , srv1348778385:86 , srv1793002439:86 , srv1796657214:86 , srv1991753655:86 , srv269806292:86 , srv415910992:86 , srv967405118:633 }
2023-07-11 20:34:11,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:11,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:11,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:11,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:11,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:11,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:11,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:11,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:11,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:11,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:11,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:11,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:11,713 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:11,713 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-11 20:34:11,715 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.342744348120761E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.833333333333334E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-11 20:34:12,209 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 627 ms to try 5000 different iterations.  Found a solution that moves 33 regions; Going from a computed imbalance of 6.342744348120761E-5 to a new imbalance of 7.618828439090763E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0036666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:12,209 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1039428461:83 , srv1389714021:83 , srv1796657214:83 , srv1991753655:83 , srv269806292:83 , srv584670424:83 , srv1022160506:84 , srv1035486473:84 , srv1090236618:84 , srv1173917895:84 , srv1179978579:84 , srv1225462505:84 , srv1228154989:84 , srv1229478515:84 , srv1251312002:84 , srv1297962039:84 , srv1348778385:84 , srv1422543242:84 , srv1423940427:84 , srv144427312:84 , srv1461003143:84 , srv1462867295:84 , srv1464193746:84 , srv1534824192:84 , srv1541312122:84 , srv1542800665:84 , srv1600313655:84 , srv1630715226:84 , srv1644734683:84 , srv1644997594:84 , srv1706758030:84 , srv1712631455:84 , srv1740862808:84 , srv1769573138:84 , srv1774644222:84 , srv178597972:84 , srv1793002439:84 , srv1811369685:84 , srv1838425314:84 , srv1846026393:84 , srv1853564796:84 , srv186425014:84 , srv1865596283:84 , srv1873124449:84 , srv1911713832:84 , srv1963907863:84 , srv1992646250:84 , srv1997142553:84 , srv2011493255:84 , srv2013506564:84 , srv2038547076:84 , srv2096161618:84 , srv209828816:84 , srv2098898801:84 , srv2116376573:84 , srv2128455015:84 , srv256219399:84 , srv263928717:84 , srv303192302:84 , srv327462984:84 , srv365406819:84 , srv384192012:84 , srv415910992:84 , srv510147641:84 , srv51031370:84 , srv530840180:84 , srv537932445:84 , srv557009583:84 , srv590851086:84 , srv598368707:84 , srv639165207:84 , srv66079082:84 , srv664463959:84 , srv669175080:84 , srv699618549:84 , srv700609737:84 , srv722346024:84 , srv740586477:84 , srv764509472:84 , srv779335656:84 , srv7826877:84 , srv784358768:84 , srv814878733:84 , srv842804091:84 , srv891762054:84 , srv897253536:84 , srv905737253:84 , srv949117111:84 , srv962460307:84 , srv966123914:84 , srv979837431:84 , srv992886659:84 , srv999856235:84 , srv1196313829:85 , srv1821747421:85 , srv275116991:85 , srv724798958:85 , srv878086313:85 , srv921539741:85 , srv967405118:684 }
2023-07-11 20:34:12,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:12,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv303192302=60, srv1740862808=29, srv764509472=82, srv1297962039=11, srv1542800665=22, srv2098898801=53, srv415910992=64, srv1541312122=21, srv1389714021=13, srv186425014=40, srv1774644222=31, srv1090236618=3, srv1997142553=47, srv1853564796=39, srv7826877=84, srv178597972=32, srv724798958=80, srv779335656=83, srv1991753655=45, srv814878733=86, srv979837431=97, srv1992646250=46, srv557009583=69, srv1796657214=34, srv784358768=85, srv1462867295=18, srv209828816=52, srv530840180=67, srv365406819=62, srv590851086=71, srv967405118=96, srv992886659=98, srv1228154989=8, srv263928717=57, srv584670424=70, srv962460307=94, srv1225462505=7, srv2013506564=49, srv1963907863=44, srv1251312002=10, srv256219399=56, srv1035486473=1, srv598368707=72, srv891762054=89, srv740586477=81, srv999856235=99, srv2011493255=48, srv2096161618=51, srv1196313829=6, srv1865596283=41, srv1422543242=14, srv1846026393=38, srv269806292=58, srv1039428461=2, srv1811369685=35, srv1644734683=25, srv1644997594=26, srv664463959=75, srv66079082=74, srv905737253=91, srv2038547076=50, srv722346024=79, srv1911713832=43, srv897253536=90, srv878086313=88, srv949117111=93, srv700609737=78, srv1229478515=9, srv639165207=73, srv1769573138=30, srv144427312=16, srv1461003143=17, srv2116376573=54, srv510147641=65, srv1712631455=28, srv966123914=95, srv1173917895=4, srv1821747421=36, srv842804091=87, srv1464193746=19, srv1793002439=33, srv2128455015=55, srv1423940427=15, srv1706758030=27, srv1348778385=12, srv384192012=63, srv1022160506=0, srv275116991=59, srv1179978579=5, srv1600313655=23, srv51031370=66, srv1534824192=20, srv1838425314=37, srv669175080=76, srv1630715226=24, srv537932445=68, srv921539741=92, srv327462984=61, srv699618549=77, srv1873124449=42} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:12,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:12,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:12,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:12,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:12,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:12,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:12,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:12,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:12,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:12,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:12,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:12,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:12,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:12,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:12,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-11 20:34:12,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-11 20:34:12,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-11 20:34:12,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-11 20:34:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-11 20:34:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-11 20:34:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-11 20:34:12,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-11 20:34:12,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-11 20:34:12,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-11 20:34:12,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-11 20:34:12,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-11 20:34:12,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-11 20:34:12,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-11 20:34:12,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-11 20:34:12,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-11 20:34:12,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-11 20:34:12,482 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:12,483 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1039428461:83 , srv1389714021:83 , srv1796657214:83 , srv1991753655:83 , srv269806292:83 , srv584670424:83 , srv1022160506:84 , srv1035486473:84 , srv1090236618:84 , srv1173917895:84 , srv1179978579:84 , srv1225462505:84 , srv1228154989:84 , srv1229478515:84 , srv1251312002:84 , srv1297962039:84 , srv1348778385:84 , srv1422543242:84 , srv1423940427:84 , srv144427312:84 , srv1461003143:84 , srv1462867295:84 , srv1464193746:84 , srv1534824192:84 , srv1541312122:84 , srv1542800665:84 , srv1600313655:84 , srv1630715226:84 , srv1644734683:84 , srv1644997594:84 , srv1706758030:84 , srv1712631455:84 , srv1740862808:84 , srv1769573138:84 , srv1774644222:84 , srv178597972:84 , srv1793002439:84 , srv1811369685:84 , srv1838425314:84 , srv1846026393:84 , srv1853564796:84 , srv186425014:84 , srv1865596283:84 , srv1873124449:84 , srv1911713832:84 , srv1963907863:84 , srv1992646250:84 , srv1997142553:84 , srv2011493255:84 , srv2013506564:84 , srv2038547076:84 , srv2096161618:84 , srv209828816:84 , srv2098898801:84 , srv2116376573:84 , srv2128455015:84 , srv256219399:84 , srv263928717:84 , srv303192302:84 , srv327462984:84 , srv365406819:84 , srv384192012:84 , srv415910992:84 , srv510147641:84 , srv51031370:84 , srv530840180:84 , srv537932445:84 , srv557009583:84 , srv590851086:84 , srv598368707:84 , srv639165207:84 , srv66079082:84 , srv664463959:84 , srv669175080:84 , srv699618549:84 , srv700609737:84 , srv722346024:84 , srv740586477:84 , srv764509472:84 , srv779335656:84 , srv7826877:84 , srv784358768:84 , srv814878733:84 , srv842804091:84 , srv891762054:84 , srv897253536:84 , srv905737253:84 , srv949117111:84 , srv962460307:84 , srv966123914:84 , srv979837431:84 , srv992886659:84 , srv999856235:84 , srv1196313829:85 , srv1821747421:85 , srv275116991:85 , srv724798958:85 , srv878086313:85 , srv921539741:85 , srv967405118:684 }
2023-07-11 20:34:12,526 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=430 (was 373) - SystemLoadAverage LEAK? -, ProcessCount=166 (was 166), AvailableMemoryMB=8518 (was 9238)
2023-07-11 20:34:12,543 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=430, ProcessCount=166, AvailableMemoryMB=8518
2023-07-11 20:34:12,544 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-11 20:34:12,544 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-11 20:34:12,566 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1302271849:84 , srv1515040870:84 , srv1537430489:84 , srv1560341373:84 , srv1566918673:84 , srv1608412700:84 , srv1623762562:84 , srv1625562633:84 , srv1806778693:84 , srv1822350267:84 , srv1829568868:84 , srv1960818315:84 , srv1974770080:84 , srv2008563947:84 , srv2082568689:84 , srv2096729955:84 , srv236038623:84 , srv246898016:84 , srv347486334:84 , srv475721353:84 , srv494804874:84 , srv543089604:84 , srv551213977:84 , srv673660813:84 , srv850527142:84 , srv858271435:84 , srv895698053:84 , srv902945677:84 , srv975789361:84 , srv929473599:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-11 20:34:12,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:34:12,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2082568689=14, srv1960818315=11, srv673660813=23, srv494804874=20, srv551213977=22, srv1537430489=2, srv246898016=17, srv2096729955=15, srv1623762562=6, srv236038623=16, srv543089604=21, srv1608412700=5, srv1974770080=12, srv1515040870=1, srv1822350267=9, srv2008563947=13, srv1625562633=7, srv1829568868=10, srv902945677=27, srv975789361=29, srv858271435=25, srv1566918673=4, srv1806778693=8, srv895698053=26, srv475721353=19, srv1560341373=3, srv850527142=24, srv929473599=28, srv347486334=18, srv1302271849=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:34:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:34:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:34:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:34:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:34:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:34:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-11 20:34:12,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1100872880988795, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:34:12,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912539924714399, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1100872880988795, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-11 20:36:12,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5593306 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912539924714399 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:36:12,613 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1302271849:90 , srv1515040870:90 , srv1537430489:90 , srv1560341373:90 , srv1566918673:90 , srv1608412700:90 , srv1623762562:90 , srv1625562633:90 , srv1806778693:90 , srv1822350267:90 , srv1829568868:90 , srv1960818315:90 , srv1974770080:90 , srv2008563947:90 , srv2082568689:90 , srv2096729955:90 , srv236038623:90 , srv246898016:90 , srv347486334:90 , srv475721353:90 , srv494804874:90 , srv543089604:90 , srv551213977:90 , srv673660813:90 , srv850527142:90 , srv858271435:90 , srv895698053:90 , srv902945677:90 , srv929473599:90 , srv975789361:90 }
2023-07-11 20:36:12,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-11 20:36:12,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2082568689=14, srv1960818315=11, srv673660813=23, srv494804874=20, srv551213977=22, srv1537430489=2, srv246898016=17, srv2096729955=15, srv1623762562=6, srv236038623=16, srv543089604=21, srv1608412700=5, srv1974770080=12, srv1515040870=1, srv1822350267=9, srv2008563947=13, srv1625562633=7, srv1829568868=10, srv902945677=27, srv975789361=29, srv858271435=25, srv1566918673=4, srv1806778693=8, srv895698053=26, srv475721353=19, srv1560341373=3, srv850527142=24, srv929473599=28, srv347486334=18, srv1302271849=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-11 20:36:12,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-11 20:36:12,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-11 20:36:12,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-11 20:36:12,632 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1302271849:90 , srv1515040870:90 , srv1537430489:90 , srv1560341373:90 , srv1566918673:90 , srv1608412700:90 , srv1623762562:90 , srv1625562633:90 , srv1806778693:90 , srv1822350267:90 , srv1829568868:90 , srv1960818315:90 , srv1974770080:90 , srv2008563947:90 , srv2082568689:90 , srv2096729955:90 , srv236038623:90 , srv246898016:90 , srv347486334:90 , srv475721353:90 , srv494804874:90 , srv543089604:90 , srv551213977:90 , srv673660813:90 , srv850527142:90 , srv858271435:90 , srv895698053:90 , srv902945677:90 , srv929473599:90 , srv975789361:90 }
2023-07-11 20:36:12,644 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=494 (was 430) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 166) - ProcessCount LEAK? -, AvailableMemoryMB=4970 (was 8518)