2023-05-27 09:53:22,602 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-27 09:53:22,892 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-27 09:53:23,000 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-27 09:53:23,059 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-27 09:53:23,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-27 09:53:23,080 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=247, ProcessCount=171, AvailableMemoryMB=5228
2023-05-27 09:53:23,081 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-27 09:53:23,082 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-27 09:53:23,227 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1037292334:84 , srv1073074460:84 , srv1080377933:84 , srv1081539293:84 , srv1090991012:84 , srv1098665165:84 , srv1106965739:84 , srv1176580140:84 , srv1182895758:84 , srv1224607269:84 , srv1237896659:84 , srv1245952674:84 , srv1271318866:84 , srv129928036:84 , srv1302869796:84 , srv1347738988:84 , srv1351526338:84 , srv1385471097:84 , srv1403236528:84 , srv1415666144:84 , srv1419816395:84 , srv1421319196:84 , srv1459609215:84 , srv1478441779:84 , srv157753694:84 , srv1597546385:84 , srv1608534895:84 , srv1637093873:84 , srv1674335430:84 , srv1695297290:84 , srv1708728747:84 , srv1718972674:84 , srv1721731235:84 , srv1725052085:84 , srv1734523099:84 , srv1736547828:84 , srv17414128:84 , srv1750402757:84 , srv1756341410:84 , srv1777399988:84 , srv1784822742:84 , srv1791205379:84 , srv1793346145:84 , srv1800613493:84 , srv1815958016:84 , srv1841264373:84 , srv1910688096:84 , srv1954539537:84 , srv1958625534:84 , srv1963329524:84 , srv1985087071:84 , srv1987620739:84 , srv1998316303:84 , srv2004261205:84 , srv2035414210:84 , srv2057522136:84 , srv2061996389:84 , srv2082316718:84 , srv2087718681:84 , srv2090175870:84 , srv2095424977:84 , srv2102211738:84 , srv248900211:84 , srv316467923:84 , srv319684370:84 , srv368211661:84 , srv386106934:84 , srv408899802:84 , srv44275260:84 , srv447254562:84 , srv452816678:84 , srv48854630:84 , srv505364243:84 , srv543341044:84 , srv558253867:84 , srv580052953:84 , srv582950692:84 , srv649556586:84 , srv67558426:84 , srv680934044:84 , srv700648056:84 , srv712393290:84 , srv731443224:84 , srv780869619:84 , srv787467380:84 , srv813249271:84 , srv822877641:84 , srv83614649:84 , srv852336732:84 , srv86518559:84 , srv892837611:84 , srv893454743:84 , srv899817452:84 , srv901245279:84 , srv915101521:84 , srv922405920:84 , srv940173847:84 , srv95494903:84 , srv999599510:84 , srv254949366:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-27 09:53:23,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:23,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813249271=86, srv940173847=97, srv1597546385=25, srv17414128=36, srv1725052085=33, srv254949366=63, srv712393290=82, srv1637093873=27, srv1998316303=52, srv543341044=74, srv1245952674=11, srv2061996389=56, srv248900211=62, srv1347738988=15, srv316467923=64, srv892837611=91, srv1176580140=7, srv452816678=71, srv1478441779=23, srv1784822742=40, srv1237896659=10, srv1695297290=29, srv2004261205=53, srv319684370=65, srv157753694=24, srv1910688096=46, srv1734523099=34, srv780869619=84, srv1841264373=45, srv822877641=87, srv505364243=73, srv1800613493=43, srv2087718681=58, srv2082316718=57, srv1608534895=26, srv1302869796=14, srv1963329524=49, srv1793346145=42, srv1073074460=1, srv83614649=88, srv86518559=90, srv2057522136=55, srv582950692=77, srv1718972674=31, srv1736547828=35, srv1777399988=39, srv1403236528=18, srv1721731235=32, srv731443224=83, srv787467380=85, srv915101521=95, srv1182895758=8, srv1419816395=20, srv95494903=98, srv368211661=66, srv1985087071=50, srv1459609215=22, srv901245279=94, srv1106965739=6, srv680934044=80, srv1098665165=5, srv1080377933=2, srv2090175870=59, srv1954539537=47, srv44275260=69, srv1750402757=37, srv67558426=79, srv1224607269=9, srv580052953=76, srv852336732=89, srv1271318866=12, srv700648056=81, srv2095424977=60, srv1037292334=0, srv893454743=92, srv1815958016=44, srv899817452=93, srv1385471097=17, srv408899802=68, srv447254562=70, srv558253867=75, srv1081539293=3, srv386106934=67, srv1958625534=48, srv1421319196=21, srv1791205379=41, srv1090991012=4, srv2102211738=61, srv129928036=13, srv649556586=78, srv1987620739=51, srv1674335430=28, srv922405920=96, srv2035414210=54, srv1415666144=19, srv1708728747=30, srv1756341410=38, srv999599510=99, srv1351526338=16, srv48854630=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:23,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:23,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:23,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:23,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-27 09:53:23,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-27 09:53:23,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-27 09:53:23,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-27 09:53:23,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-27 09:53:23,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-27 09:53:23,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-27 09:53:23,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-27 09:53:23,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-27 09:53:23,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-27 09:53:23,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-27 09:53:23,510 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:23,511 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-27 09:53:23,511 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-27 09:53:23,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 462 ms to try 5000 different iterations.  Found a solution that moves 4706 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18139730067891657. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03983004172065679, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04986401260514861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5228888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03983004172065679, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24408333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:23,833 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv129928036:82 , srv1674335430:82 , srv1841264373:82 , srv1954539537:82 , srv1421319196:83 , srv1784822742:83 , srv1793346145:83 , srv1963329524:83 , srv899817452:83 , srv1037292334:84 , srv1224607269:84 , srv1245952674:84 , srv1271318866:84 , srv1459609215:84 , srv157753694:84 , srv1608534895:84 , srv1721731235:84 , srv1756341410:84 , srv1815958016:84 , srv2102211738:84 , srv408899802:84 , srv580052953:84 , srv787467380:84 , srv922405920:84 , srv1073074460:85 , srv1708728747:85 , srv1734523099:85 , srv1750402757:85 , srv1777399988:85 , srv1910688096:85 , srv2087718681:85 , srv2095424977:85 , srv248900211:85 , srv316467923:85 , srv649556586:85 , srv822877641:85 , srv852336732:85 , srv1090991012:86 , srv1302869796:86 , srv1415666144:86 , srv1419816395:86 , srv1478441779:86 , srv1597546385:86 , srv1725052085:86 , srv1736547828:86 , srv17414128:86 , srv1958625534:86 , srv1985087071:86 , srv1987620739:86 , srv1998316303:86 , srv2004261205:86 , srv447254562:86 , srv731443224:86 , srv780869619:86 , srv892837611:86 , srv940173847:86 , srv999599510:86 , srv1098665165:87 , srv1176580140:87 , srv1800613493:87 , srv386106934:87 , srv505364243:87 , srv543341044:87 , srv558253867:87 , srv680934044:87 , srv700648056:87 , srv893454743:87 , srv901245279:87 , srv1080377933:88 , srv1081539293:88 , srv1237896659:88 , srv1403236528:88 , srv1695297290:88 , srv2057522136:88 , srv319684370:88 , srv44275260:88 , srv48854630:88 , srv67558426:88 , srv712393290:88 , srv813249271:88 , srv915101521:88 , srv1182895758:89 , srv1347738988:89 , srv1351526338:89 , srv1385471097:89 , srv1637093873:89 , srv2082316718:89 , srv452816678:89 , srv582950692:89 , srv1791205379:90 , srv2035414210:90 , srv368211661:90 , srv83614649:90 , srv95494903:90 , srv86518559:91 , srv1106965739:92 , srv1718972674:93 , srv2090175870:94 , srv2061996389:95 , srv254949366:444 }
2023-05-27 09:53:23,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:23,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813249271=86, srv940173847=97, srv1597546385=25, srv17414128=36, srv1725052085=33, srv254949366=63, srv712393290=82, srv1637093873=27, srv1998316303=52, srv543341044=74, srv1245952674=11, srv2061996389=56, srv248900211=62, srv1347738988=15, srv316467923=64, srv892837611=91, srv1176580140=7, srv452816678=71, srv1478441779=23, srv1784822742=40, srv1237896659=10, srv1695297290=29, srv2004261205=53, srv319684370=65, srv157753694=24, srv1910688096=46, srv1734523099=34, srv780869619=84, srv1841264373=45, srv822877641=87, srv505364243=73, srv1800613493=43, srv2087718681=58, srv2082316718=57, srv1608534895=26, srv1302869796=14, srv1963329524=49, srv1793346145=42, srv1073074460=1, srv83614649=88, srv86518559=90, srv2057522136=55, srv582950692=77, srv1718972674=31, srv1736547828=35, srv1777399988=39, srv1403236528=18, srv1721731235=32, srv731443224=83, srv787467380=85, srv915101521=95, srv1182895758=8, srv1419816395=20, srv95494903=98, srv368211661=66, srv1985087071=50, srv1459609215=22, srv901245279=94, srv1106965739=6, srv680934044=80, srv1098665165=5, srv1080377933=2, srv2090175870=59, srv1954539537=47, srv44275260=69, srv1750402757=37, srv67558426=79, srv1224607269=9, srv580052953=76, srv852336732=89, srv1271318866=12, srv700648056=81, srv2095424977=60, srv1037292334=0, srv893454743=92, srv1815958016=44, srv899817452=93, srv1385471097=17, srv408899802=68, srv447254562=70, srv558253867=75, srv1081539293=3, srv386106934=67, srv1958625534=48, srv1421319196=21, srv1791205379=41, srv1090991012=4, srv2102211738=61, srv129928036=13, srv649556586=78, srv1987620739=51, srv1674335430=28, srv922405920=96, srv2035414210=54, srv1415666144=19, srv1708728747=30, srv1756341410=38, srv999599510=99, srv1351526338=16, srv48854630=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:23,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:23,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:23,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:23,855 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-27 09:53:23,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-27 09:53:23,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-27 09:53:23,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-27 09:53:23,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-27 09:53:23,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-27 09:53:23,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-27 09:53:24,001 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03983004172065679, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04986401260514861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03983004172065679, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24408333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:24,001 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-27 09:53:24,002 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18136434856261688, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03983004172065679, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04986401260514861, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03983004172065679, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24408333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-27 09:53:24,398 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 511 ms to try 5000 different iterations.  Found a solution that moves 2627 regions; Going from a computed imbalance of 0.18136434856261688 to a new imbalance of 0.013851358520596069. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206453593672446); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02943920288775949, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29188888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206453593672446); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:24,410 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2090175870:78 , srv2061996389:79 , srv1718972674:80 , srv1106965739:81 , srv1182895758:81 , srv1637093873:81 , srv2035414210:81 , srv368211661:81 , srv452816678:81 , srv915101521:81 , srv1080377933:82 , srv1385471097:82 , srv1791205379:82 , srv712393290:82 , srv86518559:82 , srv1081539293:83 , srv1098665165:83 , srv1347738988:83 , srv2082316718:83 , srv67558426:83 , srv680934044:83 , srv813249271:83 , srv901245279:83 , srv1237896659:84 , srv1597546385:84 , srv1608534895:84 , srv1695297290:84 , srv2057522136:84 , srv44275260:84 , srv582950692:84 , srv700648056:84 , srv83614649:84 , srv95494903:84 , srv1415666144:85 , srv1419816395:85 , srv1708728747:85 , srv1750402757:85 , srv2004261205:85 , srv48854630:85 , srv505364243:85 , srv543341044:85 , srv649556586:85 , srv780869619:85 , srv822877641:85 , srv999599510:85 , srv1176580140:86 , srv1245952674:86 , srv1403236528:86 , srv1459609215:86 , srv17414128:86 , srv1777399988:86 , srv1910688096:86 , srv2087718681:86 , srv2095424977:86 , srv2102211738:86 , srv248900211:86 , srv558253867:86 , srv892837611:86 , srv893454743:86 , srv1090991012:87 , srv1302869796:87 , srv1351526338:87 , srv1478441779:87 , srv1721731235:87 , srv1736547828:87 , srv316467923:87 , srv319684370:87 , srv386106934:87 , srv447254562:87 , srv852336732:87 , srv940173847:87 , srv1037292334:88 , srv1073074460:88 , srv1224607269:88 , srv1271318866:88 , srv1734523099:88 , srv1800613493:88 , srv1815958016:88 , srv1958625534:88 , srv1985087071:88 , srv1987620739:88 , srv408899802:88 , srv580052953:88 , srv787467380:88 , srv922405920:88 , srv129928036:89 , srv157753694:89 , srv1674335430:89 , srv1725052085:89 , srv1756341410:89 , srv1784822742:89 , srv1793346145:89 , srv1998316303:89 , srv731443224:89 , srv1963329524:90 , srv1421319196:91 , srv1841264373:91 , srv1954539537:91 , srv899817452:92 , srv254949366:526 }
2023-05-27 09:53:24,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:24,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813249271=86, srv940173847=97, srv1597546385=25, srv17414128=36, srv1725052085=33, srv254949366=63, srv712393290=82, srv1637093873=27, srv1998316303=52, srv543341044=74, srv1245952674=11, srv2061996389=56, srv248900211=62, srv1347738988=15, srv316467923=64, srv892837611=91, srv1176580140=7, srv452816678=71, srv1478441779=23, srv1784822742=40, srv1237896659=10, srv1695297290=29, srv2004261205=53, srv319684370=65, srv157753694=24, srv1910688096=46, srv1734523099=34, srv780869619=84, srv1841264373=45, srv822877641=87, srv505364243=73, srv1800613493=43, srv2087718681=58, srv2082316718=57, srv1608534895=26, srv1302869796=14, srv1963329524=49, srv1793346145=42, srv1073074460=1, srv83614649=88, srv86518559=90, srv2057522136=55, srv582950692=77, srv1718972674=31, srv1736547828=35, srv1777399988=39, srv1403236528=18, srv1721731235=32, srv731443224=83, srv787467380=85, srv915101521=95, srv1182895758=8, srv1419816395=20, srv95494903=98, srv368211661=66, srv1985087071=50, srv1459609215=22, srv901245279=94, srv1106965739=6, srv680934044=80, srv1098665165=5, srv1080377933=2, srv2090175870=59, srv1954539537=47, srv44275260=69, srv1750402757=37, srv67558426=79, srv1224607269=9, srv580052953=76, srv852336732=89, srv1271318866=12, srv700648056=81, srv2095424977=60, srv1037292334=0, srv893454743=92, srv1815958016=44, srv899817452=93, srv1385471097=17, srv408899802=68, srv447254562=70, srv558253867=75, srv1081539293=3, srv386106934=67, srv1958625534=48, srv1421319196=21, srv1791205379=41, srv1090991012=4, srv2102211738=61, srv129928036=13, srv649556586=78, srv1987620739=51, srv1674335430=28, srv922405920=96, srv2035414210=54, srv1415666144=19, srv1708728747=30, srv1756341410=38, srv999599510=99, srv1351526338=16, srv48854630=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:24,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:24,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:24,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:24,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-27 09:53:24,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-27 09:53:24,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-27 09:53:24,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-27 09:53:24,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-27 09:53:24,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-27 09:53:24,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-27 09:53:24,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-27 09:53:24,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-27 09:53:24,633 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206453593672446); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02943920288775949, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206453593672446); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:24,633 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-27 09:53:24,633 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013832963873439392, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02206453593672446); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02943920288775949, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02206453593672446); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04766666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-27 09:53:24,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 410 ms to try 5000 different iterations.  Found a solution that moves 992 regions; Going from a computed imbalance of 0.013832963873439392 to a new imbalance of 6.155294164849427E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006124615219341183); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006979205927323111); MoveCostFunction : (multiplier=7.0, imbalance=0.11022222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006124615219341183); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:24,899 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1800613493:81 , srv1998316303:81 , srv1351526338:82 , srv1725052085:82 , srv319684370:82 , srv731443224:82 , srv899817452:82 , srv1403236528:83 , srv1954539537:83 , srv1987620739:83 , srv386106934:83 , srv582950692:83 , srv95494903:83 , srv1090991012:84 , srv1106965739:84 , srv1176580140:84 , srv1224607269:84 , srv129928036:84 , srv1421319196:84 , srv1478441779:84 , srv157753694:84 , srv1718972674:84 , srv1734523099:84 , srv1841264373:84 , srv1985087071:84 , srv2061996389:84 , srv83614649:84 , srv893454743:84 , srv1037292334:85 , srv1073074460:85 , srv1271318866:85 , srv1302869796:85 , srv1347738988:85 , srv1695297290:85 , srv1736547828:85 , srv17414128:85 , srv1756341410:85 , srv1777399988:85 , srv1815958016:85 , srv1958625534:85 , srv2004261205:85 , srv408899802:85 , srv447254562:85 , srv48854630:85 , srv558253867:85 , srv787467380:85 , srv852336732:85 , srv86518559:85 , srv892837611:85 , srv922405920:85 , srv940173847:85 , srv1098665165:86 , srv1237896659:86 , srv1385471097:86 , srv1419816395:86 , srv1674335430:86 , srv1784822742:86 , srv1791205379:86 , srv1793346145:86 , srv1910688096:86 , srv1963329524:86 , srv2035414210:86 , srv2090175870:86 , srv2095424977:86 , srv316467923:86 , srv44275260:86 , srv505364243:86 , srv543341044:86 , srv580052953:86 , srv680934044:86 , srv999599510:86 , srv1080377933:87 , srv1081539293:87 , srv1415666144:87 , srv1597546385:87 , srv1721731235:87 , srv1750402757:87 , srv2057522136:87 , srv2082316718:87 , srv2087718681:87 , srv248900211:87 , srv368211661:87 , srv452816678:87 , srv700648056:87 , srv780869619:87 , srv813249271:87 , srv1182895758:88 , srv1245952674:88 , srv1459609215:88 , srv1637093873:88 , srv1708728747:88 , srv67558426:88 , srv712393290:88 , srv822877641:88 , srv901245279:88 , srv915101521:88 , srv2102211738:89 , srv649556586:89 , srv1608534895:90 , srv254949366:542 }
2023-05-27 09:53:24,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:24,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813249271=86, srv940173847=97, srv1597546385=25, srv17414128=36, srv1725052085=33, srv254949366=63, srv712393290=82, srv1637093873=27, srv1998316303=52, srv543341044=74, srv1245952674=11, srv2061996389=56, srv248900211=62, srv1347738988=15, srv316467923=64, srv892837611=91, srv1176580140=7, srv452816678=71, srv1478441779=23, srv1784822742=40, srv1237896659=10, srv1695297290=29, srv2004261205=53, srv319684370=65, srv157753694=24, srv1910688096=46, srv1734523099=34, srv780869619=84, srv1841264373=45, srv822877641=87, srv505364243=73, srv1800613493=43, srv2087718681=58, srv2082316718=57, srv1608534895=26, srv1302869796=14, srv1963329524=49, srv1793346145=42, srv1073074460=1, srv83614649=88, srv86518559=90, srv2057522136=55, srv582950692=77, srv1718972674=31, srv1736547828=35, srv1777399988=39, srv1403236528=18, srv1721731235=32, srv731443224=83, srv787467380=85, srv915101521=95, srv1182895758=8, srv1419816395=20, srv95494903=98, srv368211661=66, srv1985087071=50, srv1459609215=22, srv901245279=94, srv1106965739=6, srv680934044=80, srv1098665165=5, srv1080377933=2, srv2090175870=59, srv1954539537=47, srv44275260=69, srv1750402757=37, srv67558426=79, srv1224607269=9, srv580052953=76, srv852336732=89, srv1271318866=12, srv700648056=81, srv2095424977=60, srv1037292334=0, srv893454743=92, srv1815958016=44, srv899817452=93, srv1385471097=17, srv408899802=68, srv447254562=70, srv558253867=75, srv1081539293=3, srv386106934=67, srv1958625534=48, srv1421319196=21, srv1791205379=41, srv1090991012=4, srv2102211738=61, srv129928036=13, srv649556586=78, srv1987620739=51, srv1674335430=28, srv922405920=96, srv2035414210=54, srv1415666144=19, srv1708728747=30, srv1756341410=38, srv999599510=99, srv1351526338=16, srv48854630=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:24,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:24,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:24,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-27 09:53:24,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-27 09:53:24,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-27 09:53:24,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-27 09:53:24,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-27 09:53:24,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-27 09:53:24,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-27 09:53:24,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-27 09:53:25,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006124615219341183); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006979205927323111); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006124615219341183); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:25,063 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-27 09:53:25,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.085832840222767E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006124615219341183); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006979205927323111); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006124615219341183); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-27 09:53:25,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 530 ms to try 5000 different iterations.  Found a solution that moves 245 regions; Going from a computed imbalance of 6.085832840222767E-4 to a new imbalance of 2.0416404742290104E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.02722222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:25,470 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2057522136:82 , srv2082316718:82 , srv1415666144:83 , srv1958625534:83 , srv1963329524:83 , srv2102211738:83 , srv558253867:83 , srv580052953:83 , srv649556586:83 , srv67558426:83 , srv822877641:83 , srv1073074460:84 , srv1080377933:84 , srv1081539293:84 , srv1182895758:84 , srv1237896659:84 , srv1245952674:84 , srv1302869796:84 , srv1351526338:84 , srv1421319196:84 , srv1637093873:84 , srv1708728747:84 , srv1721731235:84 , srv1736547828:84 , srv1756341410:84 , srv1784822742:84 , srv1791205379:84 , srv1793346145:84 , srv1985087071:84 , srv2061996389:84 , srv2087718681:84 , srv2090175870:84 , srv248900211:84 , srv316467923:84 , srv44275260:84 , srv447254562:84 , srv48854630:84 , srv543341044:84 , srv700648056:84 , srv712393290:84 , srv780869619:84 , srv813249271:84 , srv86518559:84 , srv892837611:84 , srv901245279:84 , srv915101521:84 , srv940173847:84 , srv1037292334:85 , srv1090991012:85 , srv1106965739:85 , srv1176580140:85 , srv1271318866:85 , srv1347738988:85 , srv1385471097:85 , srv1403236528:85 , srv1419816395:85 , srv1459609215:85 , srv1478441779:85 , srv157753694:85 , srv1597546385:85 , srv1608534895:85 , srv1674335430:85 , srv1695297290:85 , srv1718972674:85 , srv1725052085:85 , srv1734523099:85 , srv17414128:85 , srv1750402757:85 , srv1815958016:85 , srv1910688096:85 , srv1954539537:85 , srv1987620739:85 , srv2035414210:85 , srv368211661:85 , srv386106934:85 , srv408899802:85 , srv452816678:85 , srv505364243:85 , srv731443224:85 , srv787467380:85 , srv83614649:85 , srv852336732:85 , srv899817452:85 , srv922405920:85 , srv999599510:85 , srv1098665165:86 , srv1224607269:86 , srv1777399988:86 , srv1800613493:86 , srv1841264373:86 , srv1998316303:86 , srv2004261205:86 , srv2095424977:86 , srv319684370:86 , srv582950692:86 , srv680934044:86 , srv893454743:86 , srv95494903:86 , srv129928036:87 , srv254949366:630 }
2023-05-27 09:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv813249271=86, srv940173847=97, srv1597546385=25, srv17414128=36, srv1725052085=33, srv254949366=63, srv712393290=82, srv1637093873=27, srv1998316303=52, srv543341044=74, srv1245952674=11, srv2061996389=56, srv248900211=62, srv1347738988=15, srv316467923=64, srv892837611=91, srv1176580140=7, srv452816678=71, srv1478441779=23, srv1784822742=40, srv1237896659=10, srv1695297290=29, srv2004261205=53, srv319684370=65, srv157753694=24, srv1910688096=46, srv1734523099=34, srv780869619=84, srv1841264373=45, srv822877641=87, srv505364243=73, srv1800613493=43, srv2087718681=58, srv2082316718=57, srv1608534895=26, srv1302869796=14, srv1963329524=49, srv1793346145=42, srv1073074460=1, srv83614649=88, srv86518559=90, srv2057522136=55, srv582950692=77, srv1718972674=31, srv1736547828=35, srv1777399988=39, srv1403236528=18, srv1721731235=32, srv731443224=83, srv787467380=85, srv915101521=95, srv1182895758=8, srv1419816395=20, srv95494903=98, srv368211661=66, srv1985087071=50, srv1459609215=22, srv901245279=94, srv1106965739=6, srv680934044=80, srv1098665165=5, srv1080377933=2, srv2090175870=59, srv1954539537=47, srv44275260=69, srv1750402757=37, srv67558426=79, srv1224607269=9, srv580052953=76, srv852336732=89, srv1271318866=12, srv700648056=81, srv2095424977=60, srv1037292334=0, srv893454743=92, srv1815958016=44, srv899817452=93, srv1385471097=17, srv408899802=68, srv447254562=70, srv558253867=75, srv1081539293=3, srv386106934=67, srv1958625534=48, srv1421319196=21, srv1791205379=41, srv1090991012=4, srv2102211738=61, srv129928036=13, srv649556586=78, srv1987620739=51, srv1674335430=28, srv922405920=96, srv2035414210=54, srv1415666144=19, srv1708728747=30, srv1756341410=38, srv999599510=99, srv1351526338=16, srv48854630=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:25,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:25,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-27 09:53:25,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-27 09:53:25,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-27 09:53:25,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-27 09:53:25,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-27 09:53:25,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-27 09:53:25,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-27 09:53:25,612 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.870087807560343E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:25,612 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2057522136:82 , srv2082316718:82 , srv1415666144:83 , srv1958625534:83 , srv1963329524:83 , srv2102211738:83 , srv558253867:83 , srv580052953:83 , srv649556586:83 , srv67558426:83 , srv822877641:83 , srv1073074460:84 , srv1080377933:84 , srv1081539293:84 , srv1182895758:84 , srv1237896659:84 , srv1245952674:84 , srv1302869796:84 , srv1351526338:84 , srv1421319196:84 , srv1637093873:84 , srv1708728747:84 , srv1721731235:84 , srv1736547828:84 , srv1756341410:84 , srv1784822742:84 , srv1791205379:84 , srv1793346145:84 , srv1985087071:84 , srv2061996389:84 , srv2087718681:84 , srv2090175870:84 , srv248900211:84 , srv316467923:84 , srv44275260:84 , srv447254562:84 , srv48854630:84 , srv543341044:84 , srv700648056:84 , srv712393290:84 , srv780869619:84 , srv813249271:84 , srv86518559:84 , srv892837611:84 , srv901245279:84 , srv915101521:84 , srv940173847:84 , srv1037292334:85 , srv1090991012:85 , srv1106965739:85 , srv1176580140:85 , srv1271318866:85 , srv1347738988:85 , srv1385471097:85 , srv1403236528:85 , srv1419816395:85 , srv1459609215:85 , srv1478441779:85 , srv157753694:85 , srv1597546385:85 , srv1608534895:85 , srv1674335430:85 , srv1695297290:85 , srv1718972674:85 , srv1725052085:85 , srv1734523099:85 , srv17414128:85 , srv1750402757:85 , srv1815958016:85 , srv1910688096:85 , srv1954539537:85 , srv1987620739:85 , srv2035414210:85 , srv368211661:85 , srv386106934:85 , srv408899802:85 , srv452816678:85 , srv505364243:85 , srv731443224:85 , srv787467380:85 , srv83614649:85 , srv852336732:85 , srv899817452:85 , srv922405920:85 , srv999599510:85 , srv1098665165:86 , srv1224607269:86 , srv1777399988:86 , srv1800613493:86 , srv1841264373:86 , srv1998316303:86 , srv2004261205:86 , srv2095424977:86 , srv319684370:86 , srv582950692:86 , srv680934044:86 , srv893454743:86 , srv95494903:86 , srv129928036:87 , srv254949366:630 }
2023-05-27 09:53:25,645 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 247) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=4622 (was 5228)
2023-05-27 09:53:25,653 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=171, AvailableMemoryMB=4621
2023-05-27 09:53:25,654 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-27 09:53:25,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-27 09:53:25,664 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1122476549:84 , srv1151372861:84 , srv1281547518:84 , srv1312477078:84 , srv1377704724:84 , srv1434259631:84 , srv1502293509:84 , srv1505773201:84 , srv1507822820:84 , srv160585828:84 , srv161915568:84 , srv1626781658:84 , srv1633943838:84 , srv1644574782:84 , srv1765449921:84 , srv2080204046:84 , srv2093960481:84 , srv2120720997:84 , srv31590952:84 , srv332823744:84 , srv388597478:84 , srv402572146:84 , srv656986039:84 , srv667584253:84 , srv730309446:84 , srv82678363:84 , srv837754866:84 , srv957154304:84 , srv980985793:84 , srv230061523:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-27 09:53:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:53:25,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv837754866=27, srv160585828=9, srv332823744=20, srv1122476549=0, srv1151372861=1, srv1644574782=13, srv2093960481=16, srv31590952=19, srv1507822820=8, srv388597478=21, srv1377704724=4, srv1502293509=6, srv82678363=26, srv1505773201=7, srv957154304=28, srv2120720997=17, srv667584253=24, srv2080204046=15, srv402572146=22, srv980985793=29, srv656986039=23, srv1633943838=12, srv1434259631=5, srv1312477078=3, srv161915568=10, srv730309446=25, srv1765449921=14, srv1626781658=11, srv1281547518=2, srv230061523=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:53:25,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:53:25,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:53:25,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-27 09:53:25,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1209786129596138, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:53:25,690 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912574242923401, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1209786129596138, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-27 09:55:25,681 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5933748 different iterations.  Found a solution that moves 1857 regions; Going from a computed imbalance of 0.9912574242923401 to a new imbalance of 4.334330639506329E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:55:25,687 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1122476549:90 , srv1151372861:90 , srv1281547518:90 , srv1312477078:90 , srv1377704724:90 , srv1434259631:90 , srv1502293509:90 , srv1505773201:90 , srv1507822820:90 , srv160585828:90 , srv161915568:90 , srv1626781658:90 , srv1633943838:90 , srv1644574782:90 , srv1765449921:90 , srv2080204046:90 , srv2093960481:90 , srv2120720997:90 , srv230061523:90 , srv31590952:90 , srv332823744:90 , srv388597478:90 , srv402572146:90 , srv656986039:90 , srv667584253:90 , srv730309446:90 , srv82678363:90 , srv837754866:90 , srv957154304:90 , srv980985793:90 }
2023-05-27 09:55:25,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-27 09:55:25,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv837754866=27, srv160585828=9, srv332823744=20, srv1122476549=0, srv1151372861=1, srv1644574782=13, srv2093960481=16, srv31590952=19, srv1507822820=8, srv388597478=21, srv1377704724=4, srv1502293509=6, srv82678363=26, srv1505773201=7, srv957154304=28, srv2120720997=17, srv667584253=24, srv2080204046=15, srv402572146=22, srv980985793=29, srv656986039=23, srv1633943838=12, srv1434259631=5, srv1312477078=3, srv161915568=10, srv730309446=25, srv1765449921=14, srv1626781658=11, srv1281547518=2, srv230061523=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-27 09:55:25,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-27 09:55:25,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-27 09:55:25,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-27 09:55:25,705 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1122476549:90 , srv1151372861:90 , srv1281547518:90 , srv1312477078:90 , srv1377704724:90 , srv1434259631:90 , srv1502293509:90 , srv1505773201:90 , srv1507822820:90 , srv160585828:90 , srv161915568:90 , srv1626781658:90 , srv1633943838:90 , srv1644574782:90 , srv1765449921:90 , srv2080204046:90 , srv2093960481:90 , srv2120720997:90 , srv230061523:90 , srv31590952:90 , srv332823744:90 , srv388597478:90 , srv402572146:90 , srv656986039:90 , srv667584253:90 , srv730309446:90 , srv82678363:90 , srv837754866:90 , srv957154304:90 , srv980985793:90 }
2023-05-27 09:55:25,717 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 250), ProcessCount=169 (was 171), AvailableMemoryMB=4593 (was 4621)