2023-06-07 05:53:23,623 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-06-07 05:53:23,895 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-06-07 05:53:23,998 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-06-07 05:53:24,054 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-07 05:53:24,061 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-06-07 05:53:24,073 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=173, AvailableMemoryMB=3615
2023-06-07 05:53:24,075 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-07 05:53:24,076 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-06-07 05:53:24,199 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1046203836:84 , srv1068103669:84 , srv1090266094:84 , srv1093184688:84 , srv1118098652:84 , srv1132246161:84 , srv1136216667:84 , srv1141977926:84 , srv1150783847:84 , srv1226910571:84 , srv1237247007:84 , srv1269915858:84 , srv1277698090:84 , srv1299628942:84 , srv1340427553:84 , srv1343604085:84 , srv1349044405:84 , srv1387607206:84 , srv1410407795:84 , srv1418853855:84 , srv1429588245:84 , srv1452455:84 , srv1475592356:84 , srv1541843223:84 , srv1547892263:84 , srv1550311802:84 , srv1587524792:84 , srv1650224326:84 , srv165805688:84 , srv1662528284:84 , srv1663210367:84 , srv1673616358:84 , srv1697332880:84 , srv1719489555:84 , srv1733091134:84 , srv1753497185:84 , srv1754886583:84 , srv1766437459:84 , srv1771210621:84 , srv1781097678:84 , srv1788514514:84 , srv1839951535:84 , srv1858686234:84 , srv188647279:84 , srv191867653:84 , srv1949824040:84 , srv1984964706:84 , srv1988706585:84 , srv2073477196:84 , srv2074893841:84 , srv2118337138:84 , srv217649639:84 , srv223357411:84 , srv242504907:84 , srv250580931:84 , srv290030848:84 , srv302476199:84 , srv304925785:84 , srv317608667:84 , srv328219053:84 , srv35006497:84 , srv353948621:84 , srv356579095:84 , srv479643063:84 , srv495246489:84 , srv517954024:84 , srv520153390:84 , srv530413536:84 , srv531765423:84 , srv531768595:84 , srv537481125:84 , srv544903714:84 , srv583120217:84 , srv592079834:84 , srv598853753:84 , srv606057425:84 , srv610972744:84 , srv632661545:84 , srv633458729:84 , srv679205998:84 , srv714388821:84 , srv726018243:84 , srv756282118:84 , srv807641144:84 , srv812939753:84 , srv850050210:84 , srv850305483:84 , srv858029528:84 , srv882472625:84 , srv896650593:84 , srv898738920:84 , srv91352544:84 , srv917631822:84 , srv922562507:84 , srv93287349:84 , srv94681940:84 , srv952093891:84 , srv957895765:84 , srv974451751:84 , srv2044501774:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-06-07 05:53:24,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:24,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:24,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:24,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:24,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:24,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:24,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:24,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:24,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:24,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:24,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:24,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:24,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:24,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:24,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:24,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:24,470 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-07 05:53:24,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-07 05:53:24,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 510 ms to try 5000 different iterations.  Found a solution that moves 4737 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1711891002774499. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039046178626657, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059528638572753166, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5263333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039046178626657, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16633333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:24,855 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv35006497:81 , srv1226910571:82 , srv756282118:82 , srv917631822:82 , srv93287349:82 , srv1733091134:83 , srv520153390:83 , srv583120217:83 , srv1269915858:84 , srv1452455:84 , srv2073477196:84 , srv223357411:84 , srv531768595:84 , srv606057425:84 , srv632661545:84 , srv850050210:84 , srv974451751:84 , srv1046203836:85 , srv1068103669:85 , srv1141977926:85 , srv1237247007:85 , srv1343604085:85 , srv1475592356:85 , srv1587524792:85 , srv1719489555:85 , srv188647279:85 , srv191867653:85 , srv302476199:85 , srv317608667:85 , srv328219053:85 , srv356579095:85 , srv592079834:85 , srv610972744:85 , srv726018243:85 , srv812939753:85 , srv898738920:85 , srv1090266094:86 , srv1093184688:86 , srv1118098652:86 , srv1132246161:86 , srv1299628942:86 , srv1349044405:86 , srv1418853855:86 , srv1429588245:86 , srv1650224326:86 , srv1663210367:86 , srv1766437459:86 , srv1781097678:86 , srv290030848:86 , srv304925785:86 , srv353948621:86 , srv495246489:86 , srv531765423:86 , srv633458729:86 , srv714388821:86 , srv850305483:86 , srv858029528:86 , srv1541843223:87 , srv1788514514:87 , srv1984964706:87 , srv2074893841:87 , srv217649639:87 , srv242504907:87 , srv517954024:87 , srv537481125:87 , srv598853753:87 , srv679205998:87 , srv922562507:87 , srv1150783847:88 , srv1410407795:88 , srv1547892263:88 , srv165805688:88 , srv1662528284:88 , srv1673616358:88 , srv1771210621:88 , srv1839951535:88 , srv2118337138:88 , srv250580931:88 , srv544903714:88 , srv807641144:88 , srv896650593:88 , srv952093891:88 , srv957895765:88 , srv1387607206:89 , srv1858686234:89 , srv1988706585:89 , srv530413536:89 , srv91352544:89 , srv1136216667:90 , srv1340427553:90 , srv1550311802:90 , srv1753497185:90 , srv479643063:90 , srv882472625:90 , srv94681940:91 , srv1754886583:92 , srv1277698090:93 , srv1949824040:94 , srv1697332880:96 , srv2044501774:437 }
2023-06-07 05:53:24,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:24,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:24,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:24,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:24,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:24,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:24,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:24,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:25,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039046178626657, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059528638572753166, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039046178626657, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16633333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:25,060 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-07 05:53:25,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17115593109451077, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039046178626657, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.059528638572753166, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039046178626657, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16633333333333333, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23275, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-07 05:53:25,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 514 ms to try 5000 different iterations.  Found a solution that moves 2592 regions; Going from a computed imbalance of 0.17115593109451077 to a new imbalance of 0.014223870986040839. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022169409746760637); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037136566603882204, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.288, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022169409746760637); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:25,458 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1697332880:74 , srv1754886583:78 , srv1949824040:78 , srv1340427553:80 , srv1136216667:81 , srv1277698090:81 , srv1839951535:82 , srv2118337138:82 , srv479643063:82 , srv882472625:82 , srv1090266094:83 , srv1150783847:83 , srv1387607206:83 , srv1547892263:83 , srv1550311802:83 , srv165805688:83 , srv1753497185:83 , srv1771210621:83 , srv537481125:83 , srv94681940:83 , srv957895765:83 , srv1410407795:84 , srv1662528284:84 , srv217649639:84 , srv242504907:84 , srv495246489:84 , srv517954024:84 , srv530413536:84 , srv544903714:84 , srv896650593:84 , srv952093891:84 , srv1068103669:85 , srv1299628942:85 , srv1349044405:85 , srv1418853855:85 , srv1429588245:85 , srv1541843223:85 , srv1673616358:85 , srv1781097678:85 , srv1858686234:85 , srv188647279:85 , srv1988706585:85 , srv317608667:85 , srv356579095:85 , srv583120217:85 , srv592079834:85 , srv807641144:85 , srv858029528:85 , srv1141977926:86 , srv1237247007:86 , srv1663210367:86 , srv1788514514:86 , srv191867653:86 , srv1984964706:86 , srv2074893841:86 , srv353948621:86 , srv632661545:86 , srv726018243:86 , srv812939753:86 , srv850305483:86 , srv91352544:86 , srv1046203836:87 , srv1093184688:87 , srv1118098652:87 , srv1132246161:87 , srv1269915858:87 , srv1475592356:87 , srv1587524792:87 , srv1719489555:87 , srv1766437459:87 , srv2073477196:87 , srv250580931:87 , srv290030848:87 , srv302476199:87 , srv531765423:87 , srv598853753:87 , srv610972744:87 , srv633458729:87 , srv679205998:87 , srv714388821:87 , srv922562507:87 , srv1650224326:88 , srv223357411:88 , srv304925785:88 , srv328219053:88 , srv520153390:88 , srv531768595:88 , srv850050210:88 , srv898738920:88 , srv974451751:88 , srv1343604085:89 , srv606057425:89 , srv756282118:89 , srv1226910571:91 , srv1452455:91 , srv1733091134:92 , srv35006497:92 , srv93287349:92 , srv917631822:93 , srv2044501774:534 }
2023-06-07 05:53:25,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:25,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:25,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:25,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:25,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:25,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:25,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:25,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:25,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:25,646 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022169409746760637); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037136566603882204, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022169409746760637); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:25,647 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-07 05:53:25,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014205721414122258, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022169409746760637); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037136566603882204, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022169409746760637); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011166666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043083333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-07 05:53:25,970 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 440 ms to try 5000 different iterations.  Found a solution that moves 1026 regions; Going from a computed imbalance of 0.014205721414122258 to a new imbalance of 4.2004281812698876E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006647934776126295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.114, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006647934776126295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:25,972 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1452455:82 , srv91352544:82 , srv1277698090:83 , srv1650224326:83 , srv1733091134:83 , srv1858686234:83 , srv1988706585:83 , srv250580931:83 , srv679205998:83 , srv917631822:83 , srv922562507:83 , srv93287349:83 , srv1093184688:84 , srv1118098652:84 , srv1132246161:84 , srv1226910571:84 , srv1550311802:84 , srv1753497185:84 , srv2074893841:84 , srv304925785:84 , srv328219053:84 , srv35006497:84 , srv530413536:84 , srv598853753:84 , srv606057425:84 , srv633458729:84 , srv714388821:84 , srv807641144:84 , srv94681940:84 , srv1046203836:85 , srv1343604085:85 , srv1387607206:85 , srv1410407795:85 , srv1475592356:85 , srv1662528284:85 , srv1663210367:85 , srv1673616358:85 , srv1766437459:85 , srv1788514514:85 , srv1949824040:85 , srv1984964706:85 , srv223357411:85 , srv290030848:85 , srv302476199:85 , srv531765423:85 , srv544903714:85 , srv610972744:85 , srv882472625:85 , srv898738920:85 , srv974451751:85 , srv1141977926:86 , srv1237247007:86 , srv1418853855:86 , srv1429588245:86 , srv1541843223:86 , srv1719489555:86 , srv1771210621:86 , srv191867653:86 , srv2073477196:86 , srv242504907:86 , srv353948621:86 , srv479643063:86 , srv517954024:86 , srv531768595:86 , srv726018243:86 , srv812939753:86 , srv850050210:86 , srv850305483:86 , srv858029528:86 , srv896650593:86 , srv952093891:86 , srv1068103669:87 , srv1136216667:87 , srv1150783847:87 , srv1269915858:87 , srv1299628942:87 , srv1340427553:87 , srv1349044405:87 , srv1547892263:87 , srv1587524792:87 , srv165805688:87 , srv1697332880:87 , srv1754886583:87 , srv1781097678:87 , srv188647279:87 , srv2118337138:87 , srv217649639:87 , srv317608667:87 , srv356579095:87 , srv495246489:87 , srv520153390:87 , srv537481125:87 , srv592079834:87 , srv632661545:87 , srv756282118:87 , srv957895765:87 , srv1090266094:88 , srv1839951535:88 , srv583120217:90 , srv2044501774:546 }
2023-06-07 05:53:25,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:25,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:25,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:25,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:25,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:25,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:25,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:25,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:26,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:26,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:26,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:26,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:26,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:26,114 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006647934776126295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006647934776126295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:26,115 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-07 05:53:26,115 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.1285861257588465E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006647934776126295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006647934776126295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0036666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-07 05:53:26,424 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 406 ms to try 5000 different iterations.  Found a solution that moves 283 regions; Going from a computed imbalance of 4.1285861257588465E-4 to a new imbalance of 3.0080340186093776E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.03144444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:26,426 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1343604085:83 , srv1587524792:83 , srv217649639:83 , srv94681940:83 , srv1046203836:84 , srv1093184688:84 , srv1118098652:84 , srv1136216667:84 , srv1150783847:84 , srv1269915858:84 , srv1299628942:84 , srv1349044405:84 , srv1541843223:84 , srv1547892263:84 , srv165805688:84 , srv1673616358:84 , srv1719489555:84 , srv1733091134:84 , srv1766437459:84 , srv1781097678:84 , srv1788514514:84 , srv1839951535:84 , srv188647279:84 , srv1984964706:84 , srv290030848:84 , srv304925785:84 , srv353948621:84 , srv479643063:84 , srv520153390:84 , srv531765423:84 , srv531768595:84 , srv583120217:84 , srv598853753:84 , srv756282118:84 , srv850050210:84 , srv850305483:84 , srv896650593:84 , srv898738920:84 , srv91352544:84 , srv917631822:84 , srv952093891:84 , srv957895765:84 , srv1068103669:85 , srv1090266094:85 , srv1141977926:85 , srv1226910571:85 , srv1237247007:85 , srv1277698090:85 , srv1387607206:85 , srv1410407795:85 , srv1418853855:85 , srv1452455:85 , srv1550311802:85 , srv1650224326:85 , srv1662528284:85 , srv1663210367:85 , srv1697332880:85 , srv1753497185:85 , srv1754886583:85 , srv1771210621:85 , srv1858686234:85 , srv191867653:85 , srv1949824040:85 , srv1988706585:85 , srv2073477196:85 , srv2074893841:85 , srv2118337138:85 , srv223357411:85 , srv242504907:85 , srv250580931:85 , srv302476199:85 , srv328219053:85 , srv35006497:85 , srv356579095:85 , srv495246489:85 , srv517954024:85 , srv530413536:85 , srv544903714:85 , srv592079834:85 , srv606057425:85 , srv610972744:85 , srv632661545:85 , srv633458729:85 , srv679205998:85 , srv714388821:85 , srv726018243:85 , srv807641144:85 , srv812939753:85 , srv858029528:85 , srv882472625:85 , srv922562507:85 , srv93287349:85 , srv974451751:85 , srv1132246161:86 , srv1340427553:86 , srv1429588245:86 , srv1475592356:86 , srv317608667:86 , srv537481125:86 , srv2044501774:625 }
2023-06-07 05:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:26,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:26,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:26,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:26,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:26,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:26,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:26,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:26,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:26,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:26,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:26,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:26,568 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-06-07 05:53:26,568 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.809873183232917E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0024618298195866546); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-06-07 05:53:26,834 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 366 ms to try 5000 different iterations.  Found a solution that moves 29 regions; Going from a computed imbalance of 2.809873183232917E-5 to a new imbalance of 8.18275245717901E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0032222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:26,835 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1132246161:83 , srv1340427553:83 , srv1429588245:83 , srv1475592356:83 , srv250580931:83 , srv317608667:83 , srv537481125:83 , srv1068103669:84 , srv1090266094:84 , srv1136216667:84 , srv1141977926:84 , srv1150783847:84 , srv1226910571:84 , srv1237247007:84 , srv1269915858:84 , srv1277698090:84 , srv1299628942:84 , srv1343604085:84 , srv1349044405:84 , srv1387607206:84 , srv1410407795:84 , srv1418853855:84 , srv1452455:84 , srv1541843223:84 , srv1547892263:84 , srv1550311802:84 , srv1587524792:84 , srv1650224326:84 , srv165805688:84 , srv1662528284:84 , srv1663210367:84 , srv1673616358:84 , srv1697332880:84 , srv1719489555:84 , srv1733091134:84 , srv1753497185:84 , srv1754886583:84 , srv1766437459:84 , srv1771210621:84 , srv1781097678:84 , srv1788514514:84 , srv1839951535:84 , srv1858686234:84 , srv191867653:84 , srv1949824040:84 , srv1984964706:84 , srv1988706585:84 , srv2073477196:84 , srv2074893841:84 , srv2118337138:84 , srv223357411:84 , srv242504907:84 , srv290030848:84 , srv302476199:84 , srv304925785:84 , srv328219053:84 , srv35006497:84 , srv353948621:84 , srv356579095:84 , srv479643063:84 , srv495246489:84 , srv517954024:84 , srv520153390:84 , srv530413536:84 , srv531765423:84 , srv531768595:84 , srv544903714:84 , srv583120217:84 , srv592079834:84 , srv598853753:84 , srv606057425:84 , srv610972744:84 , srv632661545:84 , srv633458729:84 , srv679205998:84 , srv714388821:84 , srv726018243:84 , srv756282118:84 , srv807641144:84 , srv812939753:84 , srv850050210:84 , srv850305483:84 , srv858029528:84 , srv882472625:84 , srv896650593:84 , srv898738920:84 , srv917631822:84 , srv922562507:84 , srv93287349:84 , srv952093891:84 , srv957895765:84 , srv974451751:84 , srv1046203836:85 , srv1093184688:85 , srv1118098652:85 , srv188647279:85 , srv217649639:85 , srv91352544:85 , srv94681940:85 , srv2044501774:684 }
2023-06-07 05:53:26,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:26,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1949824040=45, srv520153390=67, srv1662528284=29, srv94681940=96, srv756282118=83, srv328219053=60, srv1858686234=42, srv1541843223=23, srv583120217=73, srv974451751=99, srv35006497=61, srv517954024=66, srv922562507=94, srv679205998=80, srv1839951535=41, srv1650224326=27, srv317608667=59, srv479643063=64, srv610972744=77, srv1226910571=9, srv850305483=87, srv957895765=98, srv1090266094=2, srv242504907=54, srv1663210367=30, srv2118337138=51, srv93287349=95, srv1299628942=13, srv1550311802=25, srv217649639=52, srv1429588245=20, srv353948621=62, srv91352544=92, srv1753497185=35, srv1988706585=47, srv632661545=78, srv1349044405=16, srv1343604085=15, srv1387607206=17, srv714388821=81, srv2073477196=49, srv898738920=91, srv1141977926=7, srv812939753=85, srv633458729=79, srv592079834=74, srv807641144=84, srv223357411=53, srv1697332880=32, srv1277698090=12, srv1781097678=39, srv250580931=55, srv191867653=44, srv1547892263=24, srv1766437459=37, srv188647279=43, srv1093184688=3, srv850050210=86, srv1771210621=38, srv1587524792=26, srv537481125=71, srv302476199=57, srv544903714=72, srv1673616358=31, srv1788514514=40, srv1410407795=18, srv1719489555=33, srv726018243=82, srv1136216667=6, srv606057425=76, srv882472625=89, srv2044501774=48, srv952093891=97, srv531768595=70, srv1046203836=0, srv598853753=75, srv495246489=65, srv1118098652=4, srv1733091134=34, srv1984964706=46, srv530413536=68, srv1132246161=5, srv2074893841=50, srv896650593=90, srv1269915858=11, srv1418853855=19, srv1340427553=14, srv858029528=88, srv1150783847=8, srv304925785=58, srv1068103669=1, srv1452455=21, srv356579095=63, srv1754886583=36, srv1475592356=22, srv1237247007=10, srv531765423=69, srv290030848=56, srv917631822=93, srv165805688=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:26,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:26,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-06-07 05:53:26,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-06-07 05:53:26,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-06-07 05:53:26,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-06-07 05:53:26,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-06-07 05:53:26,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-06-07 05:53:26,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-06-07 05:53:26,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.979690117040589E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:26,969 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1132246161:83 , srv1340427553:83 , srv1429588245:83 , srv1475592356:83 , srv250580931:83 , srv317608667:83 , srv537481125:83 , srv1068103669:84 , srv1090266094:84 , srv1136216667:84 , srv1141977926:84 , srv1150783847:84 , srv1226910571:84 , srv1237247007:84 , srv1269915858:84 , srv1277698090:84 , srv1299628942:84 , srv1343604085:84 , srv1349044405:84 , srv1387607206:84 , srv1410407795:84 , srv1418853855:84 , srv1452455:84 , srv1541843223:84 , srv1547892263:84 , srv1550311802:84 , srv1587524792:84 , srv1650224326:84 , srv165805688:84 , srv1662528284:84 , srv1663210367:84 , srv1673616358:84 , srv1697332880:84 , srv1719489555:84 , srv1733091134:84 , srv1753497185:84 , srv1754886583:84 , srv1766437459:84 , srv1771210621:84 , srv1781097678:84 , srv1788514514:84 , srv1839951535:84 , srv1858686234:84 , srv191867653:84 , srv1949824040:84 , srv1984964706:84 , srv1988706585:84 , srv2073477196:84 , srv2074893841:84 , srv2118337138:84 , srv223357411:84 , srv242504907:84 , srv290030848:84 , srv302476199:84 , srv304925785:84 , srv328219053:84 , srv35006497:84 , srv353948621:84 , srv356579095:84 , srv479643063:84 , srv495246489:84 , srv517954024:84 , srv520153390:84 , srv530413536:84 , srv531765423:84 , srv531768595:84 , srv544903714:84 , srv583120217:84 , srv592079834:84 , srv598853753:84 , srv606057425:84 , srv610972744:84 , srv632661545:84 , srv633458729:84 , srv679205998:84 , srv714388821:84 , srv726018243:84 , srv756282118:84 , srv807641144:84 , srv812939753:84 , srv850050210:84 , srv850305483:84 , srv858029528:84 , srv882472625:84 , srv896650593:84 , srv898738920:84 , srv917631822:84 , srv922562507:84 , srv93287349:84 , srv952093891:84 , srv957895765:84 , srv974451751:84 , srv1046203836:85 , srv1093184688:85 , srv1118098652:85 , srv188647279:85 , srv217649639:85 , srv91352544:85 , srv94681940:85 , srv2044501774:684 }
2023-06-07 05:53:26,999 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=324 (was 308) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=3084 (was 3615)
2023-06-07 05:53:27,006 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=324, ProcessCount=173, AvailableMemoryMB=3084
2023-06-07 05:53:27,007 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-06-07 05:53:27,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-06-07 05:53:27,019 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002131643:84 , srv1002709336:84 , srv1149563050:84 , srv123448107:84 , srv1298626877:84 , srv1365410042:84 , srv138517678:84 , srv1509882663:84 , srv153512058:84 , srv1644389777:84 , srv1879388713:84 , srv1915397427:84 , srv1916302399:84 , srv1982509866:84 , srv294489447:84 , srv31699594:84 , srv32685404:84 , srv345306047:84 , srv382999515:84 , srv453816217:84 , srv531748074:84 , srv564771714:84 , srv631089388:84 , srv743736615:84 , srv815375814:84 , srv819729254:84 , srv872965000:84 , srv877434301:84 , srv974052709:84 , srv789805344:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-06-07 05:53:27,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:53:27,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv31699594=15, srv1879388713=10, srv1982509866=13, srv531748074=20, srv815375814=25, srv743736615=23, srv872965000=27, srv1365410042=5, srv138517678=6, srv819729254=26, srv1298626877=4, srv1916302399=12, srv32685404=16, srv974052709=29, srv1002709336=1, srv564771714=21, srv1002131643=0, srv1644389777=9, srv382999515=18, srv453816217=19, srv631089388=22, srv877434301=28, srv789805344=24, srv1509882663=7, srv294489447=14, srv123448107=3, srv153512058=8, srv345306047=17, srv1149563050=2, srv1915397427=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:53:27,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:53:27,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:53:27,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-06-07 05:53:27,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1023297278813942, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:53:27,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912515480895462, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1023297278813942, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-06-07 05:55:27,042 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5476127 different iterations.  Found a solution that moves 1865 regions; Going from a computed imbalance of 0.9912515480895462 to a new imbalance of 4.353003038599517E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6907407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:55:27,050 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1002131643:90 , srv1002709336:90 , srv1149563050:90 , srv123448107:90 , srv1298626877:90 , srv1365410042:90 , srv138517678:90 , srv1509882663:90 , srv153512058:90 , srv1644389777:90 , srv1879388713:90 , srv1915397427:90 , srv1916302399:90 , srv1982509866:90 , srv294489447:90 , srv31699594:90 , srv32685404:90 , srv345306047:90 , srv382999515:90 , srv453816217:90 , srv531748074:90 , srv564771714:90 , srv631089388:90 , srv743736615:90 , srv789805344:90 , srv815375814:90 , srv819729254:90 , srv872965000:90 , srv877434301:90 , srv974052709:90 }
2023-06-07 05:55:27,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-06-07 05:55:27,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv31699594=15, srv1879388713=10, srv1982509866=13, srv531748074=20, srv815375814=25, srv743736615=23, srv872965000=27, srv1365410042=5, srv138517678=6, srv819729254=26, srv1298626877=4, srv1916302399=12, srv32685404=16, srv974052709=29, srv1002709336=1, srv564771714=21, srv1002131643=0, srv1644389777=9, srv382999515=18, srv453816217=19, srv631089388=22, srv877434301=28, srv789805344=24, srv1509882663=7, srv294489447=14, srv123448107=3, srv153512058=8, srv345306047=17, srv1149563050=2, srv1915397427=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-06-07 05:55:27,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-06-07 05:55:27,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-06-07 05:55:27,057 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-06-07 05:55:27,058 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-06-07 05:55:27,072 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-06-07 05:55:27,072 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1002131643:90 , srv1002709336:90 , srv1149563050:90 , srv123448107:90 , srv1298626877:90 , srv1365410042:90 , srv138517678:90 , srv1509882663:90 , srv153512058:90 , srv1644389777:90 , srv1879388713:90 , srv1915397427:90 , srv1916302399:90 , srv1982509866:90 , srv294489447:90 , srv31699594:90 , srv32685404:90 , srv345306047:90 , srv382999515:90 , srv453816217:90 , srv531748074:90 , srv564771714:90 , srv631089388:90 , srv743736615:90 , srv789805344:90 , srv815375814:90 , srv819729254:90 , srv872965000:90 , srv877434301:90 , srv974052709:90 }
2023-06-07 05:55:27,092 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=246 (was 324), ProcessCount=175 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=3466 (was 3084) - AvailableMemoryMB LEAK? -