2023-08-11 18:53:11,306 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-11 18:53:11,622 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-11 18:53:11,740 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-11 18:53:11,812 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 18:53:11,823 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-11 18:53:11,839 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=258, ProcessCount=175, AvailableMemoryMB=3409
2023-08-11 18:53:11,841 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 18:53:11,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-11 18:53:11,998 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047438906:84 , srv1048705845:84 , srv105960490:84 , srv1088214973:84 , srv1108995471:84 , srv1127323042:84 , srv1136159654:84 , srv1168365732:84 , srv1207141485:84 , srv1226919239:84 , srv1229979509:84 , srv1244050837:84 , srv1251315953:84 , srv127252593:84 , srv1339465199:84 , srv1339551708:84 , srv1378183633:84 , srv1385583439:84 , srv1393852969:84 , srv1417597899:84 , srv1426233033:84 , srv1458360725:84 , srv1485430845:84 , srv1493010457:84 , srv1494172880:84 , srv1507557290:84 , srv1508055181:84 , srv1513519629:84 , srv1516205300:84 , srv1532429804:84 , srv1535823445:84 , srv1603364919:84 , srv1632844702:84 , srv1652651875:84 , srv1676720469:84 , srv1694789721:84 , srv1711330144:84 , srv1733456847:84 , srv1735615617:84 , srv1763925134:84 , srv1776706537:84 , srv178762161:84 , srv1841154407:84 , srv1873056020:84 , srv1881262209:84 , srv1925685465:84 , srv1976991985:84 , srv2008078742:84 , srv2037879672:84 , srv20466376:84 , srv2077130178:84 , srv2142430977:84 , srv2144860042:84 , srv221955752:84 , srv252044629:84 , srv270644566:84 , srv274196949:84 , srv275425768:84 , srv305928469:84 , srv330223293:84 , srv330561116:84 , srv332995517:84 , srv343791971:84 , srv373310174:84 , srv400521316:84 , srv421268409:84 , srv422855733:84 , srv433294372:84 , srv442782145:84 , srv447945147:84 , srv509863726:84 , srv542001162:84 , srv554469719:84 , srv576562802:84 , srv579784690:84 , srv582530573:84 , srv592499564:84 , srv594210812:84 , srv615281649:84 , srv630623425:84 , srv637328290:84 , srv638322074:84 , srv666259109:84 , srv668101143:84 , srv719401519:84 , srv736384680:84 , srv748539455:84 , srv801355558:84 , srv803071796:84 , srv804052180:84 , srv809094542:84 , srv834617124:84 , srv841938564:84 , srv877847703:84 , srv886678128:84 , srv933165206:84 , srv95578762:84 , srv964124036:84 , srv96631591:84 , srv634525565:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-11 18:53:12,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:12,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:12,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:12,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:12,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:12,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:12,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:12,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:12,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:12,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:12,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:12,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:12,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:12,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:12,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:12,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:12,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:12,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:12,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:12,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:12,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:12,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:12,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:12,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:12,493 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:12,494 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 18:53:12,494 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 18:53:12,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 728 ms to try 5000 different iterations.  Found a solution that moves 4662 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18161753276152082. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042895860752344146, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042895860752344146, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.177, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:12,901 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1393852969:81 , srv221955752:81 , srv305928469:81 , srv1226919239:82 , srv1378183633:82 , srv1485430845:82 , srv1776706537:82 , srv1841154407:82 , srv330561116:82 , srv343791971:82 , srv748539455:82 , srv1088214973:83 , srv1339551708:83 , srv1493010457:83 , srv1508055181:83 , srv178762161:83 , srv2037879672:83 , srv252044629:83 , srv330223293:83 , srv841938564:83 , srv1652651875:84 , srv1676720469:84 , srv1873056020:84 , srv1976991985:84 , srv20466376:84 , srv270644566:84 , srv274196949:84 , srv804052180:84 , srv877847703:84 , srv1048705845:85 , srv1168365732:85 , srv1417597899:85 , srv1507557290:85 , srv1733456847:85 , srv1735615617:85 , srv422855733:85 , srv579784690:85 , srv582530573:85 , srv615281649:85 , srv801355558:85 , srv886678128:85 , srv1127323042:86 , srv1532429804:86 , srv1535823445:86 , srv1632844702:86 , srv1694789721:86 , srv1925685465:86 , srv2077130178:86 , srv2142430977:86 , srv332995517:86 , srv421268409:86 , srv433294372:86 , srv447945147:86 , srv509863726:86 , srv638322074:86 , srv666259109:86 , srv668101143:86 , srv96631591:86 , srv105960490:87 , srv1229979509:87 , srv1244050837:87 , srv1251315953:87 , srv1385583439:87 , srv1458360725:87 , srv1711330144:87 , srv1763925134:87 , srv2008078742:87 , srv373310174:87 , srv442782145:87 , srv594210812:87 , srv1047438906:88 , srv1207141485:88 , srv1494172880:88 , srv400521316:88 , srv554469719:88 , srv630623425:88 , srv719401519:88 , srv809094542:88 , srv834617124:88 , srv933165206:88 , srv95578762:88 , srv964124036:88 , srv1136159654:89 , srv127252593:89 , srv1513519629:89 , srv542001162:89 , srv592499564:89 , srv637328290:89 , srv736384680:89 , srv1339465199:90 , srv275425768:90 , srv803071796:90 , srv1108995471:91 , srv1516205300:91 , srv2144860042:91 , srv576562802:91 , srv1603364919:93 , srv1426233033:95 , srv1881262209:100 , srv634525565:471 }
2023-08-11 18:53:12,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:12,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:12,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:12,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:12,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:12,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:12,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:12,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:12,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:12,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:12,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:12,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:12,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:12,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:12,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:12,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:12,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:12,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:12,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:13,266 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042895860752344146, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042895860752344146, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.177, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:13,266 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 18:53:13,267 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18158488873980613, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.042895860752344146, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058910802097883005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.042895860752344146, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.177, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 18:53:13,622 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 642 ms to try 5000 different iterations.  Found a solution that moves 2695 regions; Going from a computed imbalance of 0.18158488873980613 to a new imbalance of 0.013688180449751432. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023056721807345235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03868779174466492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29944444444444446, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023056721807345235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:13,636 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1881262209:71 , srv1426233033:79 , srv1108995471:80 , srv1339465199:80 , srv1516205300:80 , srv576562802:80 , srv1207141485:81 , srv1603364919:81 , srv803071796:81 , srv1136159654:82 , srv542001162:82 , srv592499564:82 , srv127252593:83 , srv2008078742:83 , srv275425768:83 , srv554469719:83 , srv630623425:83 , srv637328290:83 , srv719401519:83 , srv736384680:83 , srv964124036:83 , srv1251315953:84 , srv1735615617:84 , srv2077130178:84 , srv2144860042:84 , srv433294372:84 , srv809094542:84 , srv933165206:84 , srv95578762:84 , srv1127323042:85 , srv1229979509:85 , srv1494172880:85 , srv1513519629:85 , srv1535823445:85 , srv1694789721:85 , srv1925685465:85 , srv2142430977:85 , srv332995517:85 , srv373310174:85 , srv400521316:85 , srv421268409:85 , srv509863726:85 , srv96631591:85 , srv1047438906:86 , srv1048705845:86 , srv105960490:86 , srv1244050837:86 , srv1385583439:86 , srv1458360725:86 , srv1507557290:86 , srv1652651875:86 , srv1676720469:86 , srv1711330144:86 , srv1763925134:86 , srv20466376:86 , srv442782145:86 , srv447945147:86 , srv582530573:86 , srv594210812:86 , srv638322074:86 , srv668101143:86 , srv834617124:86 , srv1417597899:87 , srv1493010457:87 , srv178762161:87 , srv252044629:87 , srv274196949:87 , srv422855733:87 , srv579784690:87 , srv615281649:87 , srv666259109:87 , srv1339551708:88 , srv1532429804:88 , srv2037879672:88 , srv270644566:88 , srv801355558:88 , srv877847703:88 , srv1168365732:89 , srv1485430845:89 , srv1508055181:89 , srv1632844702:89 , srv1733456847:89 , srv1776706537:89 , srv1873056020:89 , srv330223293:89 , srv748539455:89 , srv804052180:89 , srv841938564:89 , srv1088214973:90 , srv1378183633:90 , srv343791971:90 , srv886678128:90 , srv1226919239:91 , srv1841154407:91 , srv1976991985:91 , srv221955752:91 , srv1393852969:92 , srv305928469:92 , srv330561116:92 , srv634525565:508 }
2023-08-11 18:53:13,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:13,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:13,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:13,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:13,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:13,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:13,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023056721807345235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03868779174466492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023056721807345235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:13,819 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 18:53:13,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013669309656417878, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023056721807345235); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03868779174466492, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023056721807345235); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 18:53:14,243 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 550 ms to try 5000 different iterations.  Found a solution that moves 999 regions; Going from a computed imbalance of 0.013669309656417878 to a new imbalance of 8.068979711079153E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008695999168947031); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01173022609606784); MoveCostFunction : (multiplier=7.0, imbalance=0.111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008695999168947031); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:14,246 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1632844702:82 , srv1047438906:83 , srv1168365732:83 , srv1532429804:83 , srv1711330144:83 , srv1733456847:83 , srv1873056020:83 , srv1976991985:83 , srv2144860042:83 , srv834617124:83 , srv886678128:83 , srv1226919239:84 , srv1244050837:84 , srv127252593:84 , srv1385583439:84 , srv1417597899:84 , srv1426233033:84 , srv1494172880:84 , srv1603364919:84 , srv1763925134:84 , srv2142430977:84 , srv275425768:84 , srv305928469:84 , srv330561116:84 , srv841938564:84 , srv933165206:84 , srv105960490:85 , srv1088214973:85 , srv1378183633:85 , srv1393852969:85 , srv1458360725:85 , srv1507557290:85 , srv1508055181:85 , srv1513519629:85 , srv1516205300:85 , srv1694789721:85 , srv1881262209:85 , srv2037879672:85 , srv221955752:85 , srv270644566:85 , srv274196949:85 , srv332995517:85 , srv373310174:85 , srv400521316:85 , srv422855733:85 , srv442782145:85 , srv579784690:85 , srv594210812:85 , srv615281649:85 , srv638322074:85 , srv666259109:85 , srv668101143:85 , srv736384680:85 , srv748539455:85 , srv801355558:85 , srv803071796:85 , srv804052180:85 , srv877847703:85 , srv95578762:85 , srv964124036:85 , srv96631591:85 , srv1048705845:86 , srv1251315953:86 , srv1339551708:86 , srv1485430845:86 , srv1535823445:86 , srv1776706537:86 , srv1841154407:86 , srv1925685465:86 , srv2077130178:86 , srv343791971:86 , srv447945147:86 , srv509863726:86 , srv542001162:86 , srv576562802:86 , srv637328290:86 , srv809094542:86 , srv1108995471:87 , srv1127323042:87 , srv1136159654:87 , srv1207141485:87 , srv1229979509:87 , srv1493010457:87 , srv1676720469:87 , srv2008078742:87 , srv20466376:87 , srv252044629:87 , srv330223293:87 , srv421268409:87 , srv433294372:87 , srv582530573:87 , srv630623425:87 , srv719401519:87 , srv1339465199:88 , srv1735615617:88 , srv178762161:88 , srv554469719:88 , srv592499564:88 , srv1652651875:89 , srv634525565:556 }
2023-08-11 18:53:14,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:14,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:14,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:14,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:14,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:14,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:14,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:14,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:14,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:14,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:14,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:14,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:14,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:14,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:14,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:14,500 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008695999168947031); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01173022609606784); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008695999168947031); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:14,501 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 18:53:14,501 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.999028235976297E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008695999168947031); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01173022609606784); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008695999168947031); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 18:53:14,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 594 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 7.999028235976297E-4 to a new imbalance of 5.785703843002747E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:14,902 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1108995471:83 , srv1229979509:83 , srv1513519629:83 , srv1652651875:83 , srv330223293:83 , srv330561116:83 , srv554469719:83 , srv592499564:83 , srv1047438906:84 , srv1048705845:84 , srv105960490:84 , srv1088214973:84 , srv1136159654:84 , srv1339465199:84 , srv1378183633:84 , srv1393852969:84 , srv1426233033:84 , srv1493010457:84 , srv1494172880:84 , srv1603364919:84 , srv1733456847:84 , srv178762161:84 , srv1841154407:84 , srv1976991985:84 , srv2144860042:84 , srv343791971:84 , srv400521316:84 , srv421268409:84 , srv442782145:84 , srv447945147:84 , srv579784690:84 , srv582530573:84 , srv594210812:84 , srv630623425:84 , srv637328290:84 , srv666259109:84 , srv719401519:84 , srv801355558:84 , srv804052180:84 , srv809094542:84 , srv886678128:84 , srv1127323042:85 , srv1168365732:85 , srv1226919239:85 , srv1244050837:85 , srv1251315953:85 , srv1339551708:85 , srv1458360725:85 , srv1485430845:85 , srv1508055181:85 , srv1532429804:85 , srv1535823445:85 , srv1632844702:85 , srv1676720469:85 , srv1735615617:85 , srv1763925134:85 , srv1776706537:85 , srv2008078742:85 , srv20466376:85 , srv221955752:85 , srv252044629:85 , srv270644566:85 , srv275425768:85 , srv305928469:85 , srv373310174:85 , srv422855733:85 , srv433294372:85 , srv509863726:85 , srv542001162:85 , srv576562802:85 , srv615281649:85 , srv638322074:85 , srv668101143:85 , srv736384680:85 , srv834617124:85 , srv95578762:85 , srv1207141485:86 , srv127252593:86 , srv1385583439:86 , srv1417597899:86 , srv1507557290:86 , srv1516205300:86 , srv1694789721:86 , srv1711330144:86 , srv1881262209:86 , srv1925685465:86 , srv2037879672:86 , srv274196949:86 , srv332995517:86 , srv748539455:86 , srv803071796:86 , srv841938564:86 , srv877847703:86 , srv933165206:86 , srv964124036:86 , srv96631591:86 , srv1873056020:87 , srv2077130178:87 , srv2142430977:88 , srv634525565:607 }
2023-08-11 18:53:14,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:14,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:14,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:14,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:14,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:14,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:14,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:14,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:14,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:14,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:14,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:15,087 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:15,088 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-11 18:53:15,092 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.582641502864324E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-11 18:53:15,485 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 538 ms to try 5000 different iterations.  Found a solution that moves 39 regions; Going from a computed imbalance of 5.582641502864324E-5 to a new imbalance of 6.3051625764662476E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.004333333333333333); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:15,485 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1127323042:83 , srv1385583439:83 , srv1458360725:83 , srv1841154407:83 , srv1873056020:83 , srv1925685465:83 , srv2077130178:83 , srv2142430977:83 , srv877847703:83 , srv105960490:84 , srv1088214973:84 , srv1136159654:84 , srv1168365732:84 , srv1207141485:84 , srv1226919239:84 , srv1229979509:84 , srv1244050837:84 , srv1251315953:84 , srv127252593:84 , srv1339465199:84 , srv1339551708:84 , srv1393852969:84 , srv1417597899:84 , srv1426233033:84 , srv1485430845:84 , srv1507557290:84 , srv1508055181:84 , srv1513519629:84 , srv1516205300:84 , srv1532429804:84 , srv1535823445:84 , srv1603364919:84 , srv1632844702:84 , srv1652651875:84 , srv1676720469:84 , srv1694789721:84 , srv1711330144:84 , srv1735615617:84 , srv1763925134:84 , srv1776706537:84 , srv178762161:84 , srv1881262209:84 , srv1976991985:84 , srv2008078742:84 , srv2037879672:84 , srv20466376:84 , srv2144860042:84 , srv221955752:84 , srv252044629:84 , srv270644566:84 , srv274196949:84 , srv275425768:84 , srv305928469:84 , srv330223293:84 , srv332995517:84 , srv343791971:84 , srv373310174:84 , srv400521316:84 , srv421268409:84 , srv422855733:84 , srv433294372:84 , srv442782145:84 , srv447945147:84 , srv509863726:84 , srv542001162:84 , srv554469719:84 , srv576562802:84 , srv582530573:84 , srv592499564:84 , srv594210812:84 , srv615281649:84 , srv630623425:84 , srv637328290:84 , srv638322074:84 , srv666259109:84 , srv668101143:84 , srv719401519:84 , srv736384680:84 , srv748539455:84 , srv801355558:84 , srv803071796:84 , srv804052180:84 , srv809094542:84 , srv834617124:84 , srv841938564:84 , srv886678128:84 , srv933165206:84 , srv95578762:84 , srv964124036:84 , srv96631591:84 , srv1047438906:85 , srv1048705845:85 , srv1108995471:85 , srv1378183633:85 , srv1493010457:85 , srv1494172880:85 , srv1733456847:85 , srv330561116:85 , srv579784690:85 , srv634525565:684 }
2023-08-11 18:53:15,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:15,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv841938564=93, srv1632844702=32, srv400521316=64, srv1493010457=23, srv1776706537=40, srv252044629=54, srv719401519=85, srv554469719=72, srv1733456847=37, srv1841154407=42, srv1393852969=18, srv1652651875=33, srv1711330144=36, srv1339551708=15, srv1207141485=8, srv630623425=79, srv274196949=56, srv1507557290=25, srv877847703=94, srv2142430977=51, srv1088214973=3, srv638322074=82, srv1881262209=44, srv1047438906=0, srv1458360725=21, srv1516205300=28, srv594210812=77, srv886678128=95, srv1535823445=30, srv332995517=61, srv1532429804=29, srv442782145=68, srv1976991985=46, srv1048705845=1, srv1508055181=26, srv1603364919=31, srv1925685465=45, srv1513519629=27, srv1763925134=39, srv270644566=55, srv433294372=67, srv96631591=99, srv20466376=49, srv804052180=90, srv2144860042=52, srv1168365732=7, srv1339465199=14, srv2037879672=48, srv1136159654=6, srv1127323042=5, srv834617124=92, srv178762161=41, srv666259109=83, srv964124036=98, srv1226919239=9, srv1735615617=38, srv748539455=87, srv305928469=58, srv1676720469=34, srv127252593=13, srv1873056020=43, srv1694789721=35, srv809094542=91, srv275425768=57, srv1108995471=4, srv933165206=96, srv579784690=74, srv421268409=65, srv330561116=60, srv422855733=66, srv576562802=73, srv803071796=89, srv668101143=84, srv1378183633=16, srv615281649=78, srv1251315953=12, srv221955752=53, srv1426233033=20, srv509863726=70, srv447945147=69, srv801355558=88, srv1229979509=10, srv637328290=81, srv105960490=2, srv373310174=63, srv1385583439=17, srv330223293=59, srv1494172880=24, srv634525565=80, srv592499564=76, srv95578762=97, srv582530573=75, srv2008078742=47, srv343791971=62, srv1244050837=11, srv1417597899=19, srv2077130178=50, srv736384680=86, srv542001162=71, srv1485430845=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:15,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:15,503 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:15,504 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-11 18:53:15,505 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-11 18:53:15,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-11 18:53:15,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-11 18:53:15,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-11 18:53:15,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-11 18:53:15,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-11 18:53:15,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-11 18:53:15,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:15,620 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1127323042:83 , srv1385583439:83 , srv1458360725:83 , srv1841154407:83 , srv1873056020:83 , srv1925685465:83 , srv2077130178:83 , srv2142430977:83 , srv877847703:83 , srv105960490:84 , srv1088214973:84 , srv1136159654:84 , srv1168365732:84 , srv1207141485:84 , srv1226919239:84 , srv1229979509:84 , srv1244050837:84 , srv1251315953:84 , srv127252593:84 , srv1339465199:84 , srv1339551708:84 , srv1393852969:84 , srv1417597899:84 , srv1426233033:84 , srv1485430845:84 , srv1507557290:84 , srv1508055181:84 , srv1513519629:84 , srv1516205300:84 , srv1532429804:84 , srv1535823445:84 , srv1603364919:84 , srv1632844702:84 , srv1652651875:84 , srv1676720469:84 , srv1694789721:84 , srv1711330144:84 , srv1735615617:84 , srv1763925134:84 , srv1776706537:84 , srv178762161:84 , srv1881262209:84 , srv1976991985:84 , srv2008078742:84 , srv2037879672:84 , srv20466376:84 , srv2144860042:84 , srv221955752:84 , srv252044629:84 , srv270644566:84 , srv274196949:84 , srv275425768:84 , srv305928469:84 , srv330223293:84 , srv332995517:84 , srv343791971:84 , srv373310174:84 , srv400521316:84 , srv421268409:84 , srv422855733:84 , srv433294372:84 , srv442782145:84 , srv447945147:84 , srv509863726:84 , srv542001162:84 , srv554469719:84 , srv576562802:84 , srv582530573:84 , srv592499564:84 , srv594210812:84 , srv615281649:84 , srv630623425:84 , srv637328290:84 , srv638322074:84 , srv666259109:84 , srv668101143:84 , srv719401519:84 , srv736384680:84 , srv748539455:84 , srv801355558:84 , srv803071796:84 , srv804052180:84 , srv809094542:84 , srv834617124:84 , srv841938564:84 , srv886678128:84 , srv933165206:84 , srv95578762:84 , srv964124036:84 , srv96631591:84 , srv1047438906:85 , srv1048705845:85 , srv1108995471:85 , srv1378183633:85 , srv1493010457:85 , srv1494172880:85 , srv1733456847:85 , srv330561116:85 , srv579784690:85 , srv634525565:684 }
2023-08-11 18:53:15,643 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=286 (was 258) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=2522 (was 3409)
2023-08-11 18:53:15,651 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=175, AvailableMemoryMB=2522
2023-08-11 18:53:15,652 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-11 18:53:15,653 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-11 18:53:15,665 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1152659983:84 , srv1189694194:84 , srv1263498176:84 , srv1308025583:84 , srv1327715907:84 , srv1394636318:84 , srv1422551357:84 , srv1671864433:84 , srv1672567784:84 , srv167940286:84 , srv169576514:84 , srv1817880175:84 , srv1862136747:84 , srv1937207174:84 , srv1969896804:84 , srv1993227317:84 , srv2035801044:84 , srv2083655803:84 , srv217791382:84 , srv276491709:84 , srv357247318:84 , srv491359352:84 , srv519631807:84 , srv520471842:84 , srv54995185:84 , srv707846655:84 , srv754365824:84 , srv779073422:84 , srv961785041:84 , srv903698202:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-11 18:53:15,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:53:15,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1422551357=6, srv2083655803=17, srv1189694194=1, srv961785041=29, srv1152659983=0, srv1327715907=4, srv169576514=10, srv1672567784=8, srv54995185=24, srv903698202=28, srv1394636318=5, srv1969896804=14, srv1308025583=3, srv1263498176=2, srv1817880175=11, srv1993227317=15, srv519631807=22, srv707846655=25, srv1862136747=12, srv1671864433=7, srv1937207174=13, srv491359352=21, srv754365824=26, srv276491709=19, srv520471842=23, srv167940286=9, srv2035801044=16, srv779073422=27, srv357247318=20, srv217791382=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:53:15,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:53:15,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:53:15,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:53:15,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:53:15,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:53:15,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:53:15,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:53:15,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-11 18:53:15,710 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1063486865415597, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:53:15,710 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912528144502968, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1063486865415597, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-11 18:55:15,689 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5603537 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.9912528144502968 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:55:15,698 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1152659983:90 , srv1189694194:90 , srv1263498176:90 , srv1308025583:90 , srv1327715907:90 , srv1394636318:90 , srv1422551357:90 , srv1671864433:90 , srv1672567784:90 , srv167940286:90 , srv169576514:90 , srv1817880175:90 , srv1862136747:90 , srv1937207174:90 , srv1969896804:90 , srv1993227317:90 , srv2035801044:90 , srv2083655803:90 , srv217791382:90 , srv276491709:90 , srv357247318:90 , srv491359352:90 , srv519631807:90 , srv520471842:90 , srv54995185:90 , srv707846655:90 , srv754365824:90 , srv779073422:90 , srv903698202:90 , srv961785041:90 }
2023-08-11 18:55:15,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-11 18:55:15,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1422551357=6, srv2083655803=17, srv1189694194=1, srv961785041=29, srv1152659983=0, srv1327715907=4, srv169576514=10, srv1672567784=8, srv54995185=24, srv903698202=28, srv1394636318=5, srv1969896804=14, srv1308025583=3, srv1263498176=2, srv1817880175=11, srv1993227317=15, srv519631807=22, srv707846655=25, srv1862136747=12, srv1671864433=7, srv1937207174=13, srv491359352=21, srv754365824=26, srv276491709=19, srv520471842=23, srv167940286=9, srv2035801044=16, srv779073422=27, srv357247318=20, srv217791382=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-11 18:55:15,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-11 18:55:15,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-11 18:55:15,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-11 18:55:15,725 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-11 18:55:15,725 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1152659983:90 , srv1189694194:90 , srv1263498176:90 , srv1308025583:90 , srv1327715907:90 , srv1394636318:90 , srv1422551357:90 , srv1671864433:90 , srv1672567784:90 , srv167940286:90 , srv169576514:90 , srv1817880175:90 , srv1862136747:90 , srv1937207174:90 , srv1969896804:90 , srv1993227317:90 , srv2035801044:90 , srv2083655803:90 , srv217791382:90 , srv276491709:90 , srv357247318:90 , srv491359352:90 , srv519631807:90 , srv520471842:90 , srv54995185:90 , srv707846655:90 , srv754365824:90 , srv779073422:90 , srv903698202:90 , srv961785041:90 }
2023-08-11 18:55:15,743 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=242 (was 286), ProcessCount=176 (was 175) - ProcessCount LEAK? -, AvailableMemoryMB=2448 (was 2522)