2023-09-03 04:53:11,841 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-03 04:53:12,177 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-03 04:53:12,297 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-03 04:53:12,369 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-03 04:53:12,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-03 04:53:12,393 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=167, AvailableMemoryMB=10173
2023-09-03 04:53:12,395 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-03 04:53:12,396 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-03 04:53:12,532 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100661661:84 , srv1009763612:84 , srv1059571480:84 , srv1084677919:84 , srv112076267:84 , srv1123013373:84 , srv1203016330:84 , srv1205312536:84 , srv1252651203:84 , srv125527105:84 , srv127097318:84 , srv1282158501:84 , srv1286984799:84 , srv1313923231:84 , srv1410619343:84 , srv1424228999:84 , srv1427883177:84 , srv1489914287:84 , srv1502532236:84 , srv1509808014:84 , srv1524062782:84 , srv1534153028:84 , srv1564817268:84 , srv1580143188:84 , srv1646973520:84 , srv1653091550:84 , srv1656151083:84 , srv1684147443:84 , srv1694373962:84 , srv1695487711:84 , srv1715253334:84 , srv1720158119:84 , srv1730872338:84 , srv1732830134:84 , srv1739315424:84 , srv1761297745:84 , srv1763630108:84 , srv1781598568:84 , srv183406625:84 , srv1889067360:84 , srv1894712174:84 , srv196882585:84 , srv2002486600:84 , srv2027253694:84 , srv2048843447:84 , srv2066193745:84 , srv2067063749:84 , srv2071874964:84 , srv2103351215:84 , srv2105616386:84 , srv2114292162:84 , srv216050372:84 , srv219304961:84 , srv221204189:84 , srv305740292:84 , srv310850331:84 , srv332296835:84 , srv336607912:84 , srv349292544:84 , srv352689594:84 , srv361799242:84 , srv373823638:84 , srv40197784:84 , srv410288474:84 , srv410972332:84 , srv412490184:84 , srv426205217:84 , srv428448659:84 , srv465859972:84 , srv480872937:84 , srv508747240:84 , srv510737750:84 , srv513185940:84 , srv569429798:84 , srv583662904:84 , srv622781820:84 , srv625102265:84 , srv652822620:84 , srv654005182:84 , srv654542222:84 , srv667248053:84 , srv686494131:84 , srv68802965:84 , srv711612761:84 , srv718819783:84 , srv731480980:84 , srv738470000:84 , srv76125276:84 , srv781696859:84 , srv815901264:84 , srv827181082:84 , srv858994491:84 , srv863466425:84 , srv892063258:84 , srv952420239:84 , srv985609525:84 , srv987791531:84 , srv992029687:84 , srv995192244:84 , srv1932832093:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-03 04:53:12,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:12,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:12,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:12,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:12,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:12,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:12,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:12,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:12,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:12,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:12,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:12,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:12,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:12,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:12,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:12,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:12,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:12,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:12,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:12,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:12,837 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-03 04:53:12,837 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-03 04:53:13,691 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1026 ms to try 5000 different iterations.  Found a solution that moves 4693 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17997245816845178. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039349366132623805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06053076130343202, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5214444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039349366132623805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:13,731 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1720158119:82 , srv127097318:83 , srv1502532236:83 , srv1580143188:83 , srv428448659:83 , srv622781820:83 , srv992029687:83 , srv1059571480:84 , srv1410619343:84 , srv1524062782:84 , srv1694373962:84 , srv1739315424:84 , srv1763630108:84 , srv332296835:84 , srv373823638:84 , srv508747240:84 , srv583662904:84 , srv652822620:84 , srv711612761:84 , srv985609525:84 , srv1009763612:85 , srv112076267:85 , srv1205312536:85 , srv1286984799:85 , srv1715253334:85 , srv2002486600:85 , srv2027253694:85 , srv2067063749:85 , srv2103351215:85 , srv2114292162:85 , srv219304961:85 , srv310850331:85 , srv352689594:85 , srv686494131:85 , srv738470000:85 , srv1252651203:86 , srv1730872338:86 , srv1894712174:86 , srv2071874964:86 , srv40197784:86 , srv410288474:86 , srv480872937:86 , srv569429798:86 , srv68802965:86 , srv718819783:86 , srv76125276:86 , srv863466425:86 , srv100661661:87 , srv125527105:87 , srv1509808014:87 , srv1564817268:87 , srv1646973520:87 , srv1684147443:87 , srv1732830134:87 , srv1761297745:87 , srv196882585:87 , srv2105616386:87 , srv216050372:87 , srv305740292:87 , srv410972332:87 , srv510737750:87 , srv625102265:87 , srv654005182:87 , srv654542222:87 , srv731480980:87 , srv781696859:87 , srv827181082:87 , srv858994491:87 , srv995192244:87 , srv1203016330:88 , srv1282158501:88 , srv1313923231:88 , srv1427883177:88 , srv1489914287:88 , srv1534153028:88 , srv1653091550:88 , srv183406625:88 , srv1889067360:88 , srv221204189:88 , srv336607912:88 , srv426205217:88 , srv815901264:88 , srv987791531:88 , srv1084677919:89 , srv1123013373:89 , srv1424228999:89 , srv1656151083:89 , srv1695487711:89 , srv1781598568:89 , srv349292544:89 , srv465859972:89 , srv667248053:89 , srv952420239:89 , srv2048843447:90 , srv2066193745:90 , srv412490184:91 , srv513185940:91 , srv892063258:91 , srv361799242:92 , srv1932832093:440 }
2023-09-03 04:53:13,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:13,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:13,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:13,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:13,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:13,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:13,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:13,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:13,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:13,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:13,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:13,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:13,927 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039349366132623805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06053076130343202, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039349366132623805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:13,927 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-03 04:53:13,928 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17993959708009763, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039349366132623805, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06053076130343202, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039349366132623805, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17558333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23775, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-03 04:53:14,270 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 2599 regions; Going from a computed imbalance of 0.17993959708009763 to a new imbalance of 0.016117720392546983. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022048705283166246); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03964771921067219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2887777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022048705283166246); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04733333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:14,279 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2048843447:80 , srv412490184:80 , srv1695487711:81 , srv667248053:81 , srv2105616386:82 , srv349292544:82 , srv361799242:82 , srv465859972:82 , srv513185940:82 , srv952420239:82 , srv987791531:82 , srv1123013373:83 , srv1282158501:83 , srv1424228999:83 , srv1427883177:83 , srv1534153028:83 , srv1653091550:83 , srv1684147443:83 , srv1781598568:83 , srv1889067360:83 , srv2066193745:83 , srv815901264:83 , srv892063258:83 , srv125527105:84 , srv1564817268:84 , srv1732830134:84 , srv221204189:84 , srv336607912:84 , srv40197784:84 , srv410972332:84 , srv426205217:84 , srv480872937:84 , srv68802965:84 , srv76125276:84 , srv781696859:84 , srv995192244:84 , srv1313923231:85 , srv1489914287:85 , srv1509808014:85 , srv1646973520:85 , srv1656151083:85 , srv1761297745:85 , srv183406625:85 , srv2071874964:85 , srv305740292:85 , srv569429798:85 , srv654005182:85 , srv718819783:85 , srv858994491:85 , srv863466425:85 , srv100661661:86 , srv1084677919:86 , srv1203016330:86 , srv1252651203:86 , srv1286984799:86 , srv196882585:86 , srv410288474:86 , srv583662904:86 , srv652822620:86 , srv654542222:86 , srv827181082:86 , srv1009763612:87 , srv1410619343:87 , srv1580143188:87 , srv1730872338:87 , srv1739315424:87 , srv1763630108:87 , srv1894712174:87 , srv2067063749:87 , srv2103351215:87 , srv216050372:87 , srv352689594:87 , srv508747240:87 , srv510737750:87 , srv625102265:87 , srv731480980:87 , srv1059571480:88 , srv112076267:88 , srv1205312536:88 , srv1715253334:88 , srv2002486600:88 , srv2027253694:88 , srv2114292162:88 , srv219304961:88 , srv310850331:88 , srv332296835:88 , srv738470000:88 , srv1502532236:89 , srv1524062782:89 , srv1694373962:89 , srv428448659:89 , srv686494131:89 , srv711612761:89 , srv992029687:89 , srv127097318:90 , srv1720158119:90 , srv373823638:90 , srv985609525:91 , srv622781820:93 , srv1932832093:530 }
2023-09-03 04:53:14,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:14,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:14,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:14,428 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022048705283166246); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03964771921067219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022048705283166246); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04733333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:14,429 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-03 04:53:14,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016099521805580785, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022048705283166246); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03964771921067219, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022048705283166246); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04733333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-03 04:53:14,759 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 431 ms to try 5000 different iterations.  Found a solution that moves 1019 regions; Going from a computed imbalance of 0.016099521805580785 to a new imbalance of 8.612019704805097E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.11322222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:14,762 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv622781820:81 , srv1084677919:82 , srv216050372:83 , srv985609525:83 , srv100661661:84 , srv112076267:84 , srv1203016330:84 , srv127097318:84 , srv1656151083:84 , srv1694373962:84 , srv1715253334:84 , srv1730872338:84 , srv183406625:84 , srv1894712174:84 , srv196882585:84 , srv2002486600:84 , srv510737750:84 , srv513185940:84 , srv625102265:84 , srv686494131:84 , srv711612761:84 , srv731480980:84 , srv827181082:84 , srv892063258:84 , srv1009763612:85 , srv1059571480:85 , srv1123013373:85 , srv1205312536:85 , srv1252651203:85 , srv1313923231:85 , srv1424228999:85 , srv1489914287:85 , srv1502532236:85 , srv1524062782:85 , srv1646973520:85 , srv1720158119:85 , srv1761297745:85 , srv1781598568:85 , srv2027253694:85 , srv2066193745:85 , srv2103351215:85 , srv2114292162:85 , srv219304961:85 , srv310850331:85 , srv332296835:85 , srv336607912:85 , srv361799242:85 , srv373823638:85 , srv410972332:85 , srv426205217:85 , srv654542222:85 , srv738470000:85 , srv863466425:85 , srv992029687:85 , srv125527105:86 , srv1286984799:86 , srv1410619343:86 , srv1427883177:86 , srv1509808014:86 , srv1653091550:86 , srv1732830134:86 , srv1739315424:86 , srv1763630108:86 , srv2067063749:86 , srv2071874964:86 , srv221204189:86 , srv305740292:86 , srv349292544:86 , srv352689594:86 , srv410288474:86 , srv428448659:86 , srv465859972:86 , srv508747240:86 , srv569429798:86 , srv654005182:86 , srv718819783:86 , srv781696859:86 , srv858994491:86 , srv1282158501:87 , srv1534153028:87 , srv1564817268:87 , srv1580143188:87 , srv1684147443:87 , srv1889067360:87 , srv2048843447:87 , srv40197784:87 , srv412490184:87 , srv76125276:87 , srv815901264:87 , srv952420239:87 , srv1695487711:88 , srv480872937:88 , srv583662904:88 , srv652822620:88 , srv667248053:88 , srv68802965:88 , srv987791531:88 , srv995192244:88 , srv2105616386:89 , srv1932832093:540 }
2023-09-03 04:53:14,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:14,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:14,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:14,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:14,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:14,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:14,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:14,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:14,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:14,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:14,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:14,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:14,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:14,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:14,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:14,911 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:14,912 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-03 04:53:14,912 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.540667799770251E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0067834882385377025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004666666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-03 04:53:15,318 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 512 ms to try 5000 different iterations.  Found a solution that moves 222 regions; Going from a computed imbalance of 8.540667799770251E-4 to a new imbalance of 2.6399946856197637E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.024666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:15,320 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv361799242:82 , srv1534153028:83 , srv373823638:83 , srv995192244:83 , srv1123013373:84 , srv1203016330:84 , srv1205312536:84 , srv125527105:84 , srv1282158501:84 , srv1313923231:84 , srv1489914287:84 , srv1509808014:84 , srv1524062782:84 , srv1564817268:84 , srv1656151083:84 , srv1695487711:84 , srv1781598568:84 , srv1889067360:84 , srv2027253694:84 , srv2066193745:84 , srv2067063749:84 , srv2105616386:84 , srv2114292162:84 , srv219304961:84 , srv221204189:84 , srv305740292:84 , srv310850331:84 , srv352689594:84 , srv428448659:84 , srv480872937:84 , srv510737750:84 , srv583662904:84 , srv625102265:84 , srv652822620:84 , srv654005182:84 , srv654542222:84 , srv667248053:84 , srv686494131:84 , srv68802965:84 , srv718819783:84 , srv731480980:84 , srv738470000:84 , srv76125276:84 , srv781696859:84 , srv815901264:84 , srv858994491:84 , srv952420239:84 , srv985609525:84 , srv987791531:84 , srv100661661:85 , srv1009763612:85 , srv1084677919:85 , srv112076267:85 , srv1252651203:85 , srv127097318:85 , srv1286984799:85 , srv1410619343:85 , srv1424228999:85 , srv1427883177:85 , srv1502532236:85 , srv1580143188:85 , srv1646973520:85 , srv1653091550:85 , srv1684147443:85 , srv1694373962:85 , srv1715253334:85 , srv1720158119:85 , srv1730872338:85 , srv1732830134:85 , srv1761297745:85 , srv1894712174:85 , srv196882585:85 , srv2002486600:85 , srv2048843447:85 , srv2071874964:85 , srv2103351215:85 , srv216050372:85 , srv332296835:85 , srv336607912:85 , srv349292544:85 , srv40197784:85 , srv410288474:85 , srv412490184:85 , srv426205217:85 , srv465859972:85 , srv508747240:85 , srv513185940:85 , srv569429798:85 , srv622781820:85 , srv711612761:85 , srv892063258:85 , srv992029687:85 , srv1059571480:86 , srv1739315424:86 , srv1763630108:86 , srv183406625:86 , srv410972332:86 , srv827181082:86 , srv863466425:86 , srv1932832093:632 }
2023-09-03 04:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:15,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:15,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:15,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:15,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:15,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:15,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:15,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:15,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:15,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:15,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:15,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:15,466 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-03 04:53:15,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.4845469631689714E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-03 04:53:15,760 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 2.4845469631689714E-5 to a new imbalance of 6.200130331567064E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:15,761 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1059571480:83 , srv1739315424:83 , srv1763630108:83 , srv183406625:83 , srv410288474:83 , srv412490184:83 , srv827181082:83 , srv892063258:83 , srv100661661:84 , srv1009763612:84 , srv1084677919:84 , srv112076267:84 , srv1203016330:84 , srv1205312536:84 , srv1252651203:84 , srv127097318:84 , srv1282158501:84 , srv1286984799:84 , srv1313923231:84 , srv1410619343:84 , srv1424228999:84 , srv1427883177:84 , srv1489914287:84 , srv1502532236:84 , srv1509808014:84 , srv1524062782:84 , srv1564817268:84 , srv1580143188:84 , srv1646973520:84 , srv1653091550:84 , srv1656151083:84 , srv1684147443:84 , srv1694373962:84 , srv1695487711:84 , srv1715253334:84 , srv1720158119:84 , srv1730872338:84 , srv1732830134:84 , srv1761297745:84 , srv1889067360:84 , srv1894712174:84 , srv196882585:84 , srv2002486600:84 , srv2027253694:84 , srv2048843447:84 , srv2066193745:84 , srv2067063749:84 , srv2071874964:84 , srv2103351215:84 , srv2105616386:84 , srv2114292162:84 , srv216050372:84 , srv219304961:84 , srv221204189:84 , srv305740292:84 , srv310850331:84 , srv332296835:84 , srv336607912:84 , srv349292544:84 , srv352689594:84 , srv373823638:84 , srv40197784:84 , srv410972332:84 , srv426205217:84 , srv428448659:84 , srv465859972:84 , srv480872937:84 , srv508747240:84 , srv510737750:84 , srv513185940:84 , srv569429798:84 , srv583662904:84 , srv622781820:84 , srv625102265:84 , srv652822620:84 , srv654005182:84 , srv654542222:84 , srv667248053:84 , srv686494131:84 , srv68802965:84 , srv711612761:84 , srv731480980:84 , srv738470000:84 , srv815901264:84 , srv858994491:84 , srv863466425:84 , srv952420239:84 , srv985609525:84 , srv987791531:84 , srv992029687:84 , srv995192244:84 , srv1123013373:85 , srv125527105:85 , srv1534153028:85 , srv1781598568:85 , srv361799242:85 , srv718819783:85 , srv76125276:85 , srv781696859:85 , srv1932832093:684 }
2023-09-03 04:53:15,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:15,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1410619343=14, srv127097318=10, srv349292544=59, srv1730872338=32, srv508747240=71, srv1646973520=24, srv336607912=58, srv76125276=88, srv125527105=9, srv827181082=91, srv1656151083=26, srv1427883177=16, srv361799242=61, srv2067063749=47, srv1282158501=11, srv718819783=85, srv426205217=67, srv332296835=57, srv1932832093=41, srv412490184=66, srv625102265=77, srv2027253694=44, srv410288474=64, srv465859972=69, srv2066193745=46, srv1715253334=30, srv1502532236=18, srv373823638=62, srv815901264=90, srv1059571480=2, srv1761297745=35, srv731480980=86, srv995192244=99, srv892063258=94, srv1286984799=12, srv100661661=0, srv1580143188=23, srv2103351215=49, srv216050372=52, srv68802965=83, srv1534153028=21, srv1489914287=17, srv221204189=54, srv1205312536=7, srv1695487711=29, srv183406625=38, srv2114292162=51, srv781696859=89, srv1009763612=1, srv1684147443=27, srv1524062782=20, srv1732830134=33, srv711612761=84, srv410972332=65, srv1424228999=15, srv1123013373=5, srv952420239=95, srv622781820=76, srv863466425=93, srv686494131=82, srv305740292=55, srv1084677919=3, srv738470000=87, srv2002486600=43, srv652822620=78, srv1894712174=40, srv1781598568=37, srv654542222=80, srv2048843447=45, srv1313923231=13, srv1653091550=25, srv992029687=98, srv40197784=63, srv1720158119=31, srv987791531=97, srv112076267=4, srv858994491=92, srv1694373962=28, srv1509808014=19, srv1889067360=39, srv1252651203=8, srv1739315424=34, srv1564817268=22, srv667248053=81, srv2105616386=50, srv569429798=74, srv583662904=75, srv219304961=53, srv654005182=79, srv510737750=72, srv513185940=73, srv1763630108=36, srv1203016330=6, srv310850331=56, srv196882585=42, srv985609525=96, srv428448659=68, srv2071874964=48, srv352689594=60, srv480872937=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:15,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:15,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:15,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-03 04:53:15,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-03 04:53:15,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-03 04:53:15,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-03 04:53:15,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-03 04:53:15,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-03 04:53:15,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-03 04:53:15,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-03 04:53:15,932 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:15,932 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1059571480:83 , srv1739315424:83 , srv1763630108:83 , srv183406625:83 , srv410288474:83 , srv412490184:83 , srv827181082:83 , srv892063258:83 , srv100661661:84 , srv1009763612:84 , srv1084677919:84 , srv112076267:84 , srv1203016330:84 , srv1205312536:84 , srv1252651203:84 , srv127097318:84 , srv1282158501:84 , srv1286984799:84 , srv1313923231:84 , srv1410619343:84 , srv1424228999:84 , srv1427883177:84 , srv1489914287:84 , srv1502532236:84 , srv1509808014:84 , srv1524062782:84 , srv1564817268:84 , srv1580143188:84 , srv1646973520:84 , srv1653091550:84 , srv1656151083:84 , srv1684147443:84 , srv1694373962:84 , srv1695487711:84 , srv1715253334:84 , srv1720158119:84 , srv1730872338:84 , srv1732830134:84 , srv1761297745:84 , srv1889067360:84 , srv1894712174:84 , srv196882585:84 , srv2002486600:84 , srv2027253694:84 , srv2048843447:84 , srv2066193745:84 , srv2067063749:84 , srv2071874964:84 , srv2103351215:84 , srv2105616386:84 , srv2114292162:84 , srv216050372:84 , srv219304961:84 , srv221204189:84 , srv305740292:84 , srv310850331:84 , srv332296835:84 , srv336607912:84 , srv349292544:84 , srv352689594:84 , srv373823638:84 , srv40197784:84 , srv410972332:84 , srv426205217:84 , srv428448659:84 , srv465859972:84 , srv480872937:84 , srv508747240:84 , srv510737750:84 , srv513185940:84 , srv569429798:84 , srv583662904:84 , srv622781820:84 , srv625102265:84 , srv652822620:84 , srv654005182:84 , srv654542222:84 , srv667248053:84 , srv686494131:84 , srv68802965:84 , srv711612761:84 , srv731480980:84 , srv738470000:84 , srv815901264:84 , srv858994491:84 , srv863466425:84 , srv952420239:84 , srv985609525:84 , srv987791531:84 , srv992029687:84 , srv995192244:84 , srv1123013373:85 , srv125527105:85 , srv1534153028:85 , srv1781598568:85 , srv361799242:85 , srv718819783:85 , srv76125276:85 , srv781696859:85 , srv1932832093:684 }
2023-09-03 04:53:15,958 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=252 (was 231) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=9301 (was 10173)
2023-09-03 04:53:15,968 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=167, AvailableMemoryMB=9301
2023-09-03 04:53:15,969 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-03 04:53:15,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-03 04:53:15,980 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004567771:84 , srv1095969786:84 , srv1124747107:84 , srv1162263749:84 , srv1290942991:84 , srv1292953176:84 , srv1323444269:84 , srv1353295036:84 , srv1542396177:84 , srv1804751027:84 , srv1881613080:84 , srv1952235883:84 , srv19806165:84 , srv2093554435:84 , srv2140827640:84 , srv265639047:84 , srv270336190:84 , srv302207975:84 , srv324857561:84 , srv350489615:84 , srv563970745:84 , srv66413351:84 , srv688362221:84 , srv777987378:84 , srv791154938:84 , srv809086922:84 , srv951152946:84 , srv980020174:84 , srv992518095:84 , srv1222811665:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-03 04:53:15,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:53:15,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv324857561=19, srv1095969786=1, srv1323444269=7, srv1542396177=9, srv1881613080=11, srv1124747107=2, srv791154938=25, srv2093554435=14, srv980020174=28, srv951152946=27, srv1353295036=8, srv777987378=24, srv2140827640=15, srv66413351=22, srv1162263749=3, srv265639047=16, srv1292953176=6, srv1804751027=10, srv688362221=23, srv1952235883=12, srv563970745=21, srv350489615=20, srv1222811665=4, srv1004567771=0, srv270336190=17, srv19806165=13, srv992518095=29, srv1290942991=5, srv302207975=18, srv809086922=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:53:15,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:53:15,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:53:15,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:53:15,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:53:15,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:53:15,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-03 04:53:16,021 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0800834193495552, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:53:16,021 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912445383503686, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0800834193495552, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-03 04:55:16,001 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6352624 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912445383503686 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:55:16,019 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004567771:90 , srv1095969786:90 , srv1124747107:90 , srv1162263749:90 , srv1222811665:90 , srv1290942991:90 , srv1292953176:90 , srv1323444269:90 , srv1353295036:90 , srv1542396177:90 , srv1804751027:90 , srv1881613080:90 , srv1952235883:90 , srv19806165:90 , srv2093554435:90 , srv2140827640:90 , srv265639047:90 , srv270336190:90 , srv302207975:90 , srv324857561:90 , srv350489615:90 , srv563970745:90 , srv66413351:90 , srv688362221:90 , srv777987378:90 , srv791154938:90 , srv809086922:90 , srv951152946:90 , srv980020174:90 , srv992518095:90 }
2023-09-03 04:55:16,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-03 04:55:16,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv324857561=19, srv1095969786=1, srv1323444269=7, srv1542396177=9, srv1881613080=11, srv1124747107=2, srv791154938=25, srv2093554435=14, srv980020174=28, srv951152946=27, srv1353295036=8, srv777987378=24, srv2140827640=15, srv66413351=22, srv1162263749=3, srv265639047=16, srv1292953176=6, srv1804751027=10, srv688362221=23, srv1952235883=12, srv563970745=21, srv350489615=20, srv1222811665=4, srv1004567771=0, srv270336190=17, srv19806165=13, srv992518095=29, srv1290942991=5, srv302207975=18, srv809086922=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-03 04:55:16,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-03 04:55:16,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-03 04:55:16,038 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-03 04:55:16,039 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004567771:90 , srv1095969786:90 , srv1124747107:90 , srv1162263749:90 , srv1222811665:90 , srv1290942991:90 , srv1292953176:90 , srv1323444269:90 , srv1353295036:90 , srv1542396177:90 , srv1804751027:90 , srv1881613080:90 , srv1952235883:90 , srv19806165:90 , srv2093554435:90 , srv2140827640:90 , srv265639047:90 , srv270336190:90 , srv302207975:90 , srv324857561:90 , srv350489615:90 , srv563970745:90 , srv66413351:90 , srv688362221:90 , srv777987378:90 , srv791154938:90 , srv809086922:90 , srv951152946:90 , srv980020174:90 , srv992518095:90 }
2023-09-03 04:55:16,051 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 252), ProcessCount=167 (was 167), AvailableMemoryMB=9034 (was 9301)