2023-11-04 17:55:26,853 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-04 17:55:27,144 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-04 17:55:27,242 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-04 17:55:27,301 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-04 17:55:27,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-04 17:55:27,322 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=146, ProcessCount=170, AvailableMemoryMB=6527
2023-11-04 17:55:27,325 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-04 17:55:27,325 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-04 17:55:27,450 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017306779:84 , srv1035555615:84 , srv1038325126:84 , srv1087605673:84 , srv1111890536:84 , srv1116641820:84 , srv113448828:84 , srv1148584163:84 , srv1181182758:84 , srv1185212772:84 , srv1244991265:84 , srv1250394368:84 , srv1251535175:84 , srv1272779064:84 , srv1286449499:84 , srv129009956:84 , srv1304637531:84 , srv1329406337:84 , srv1336356916:84 , srv1350508274:84 , srv1394920619:84 , srv1404506237:84 , srv1432388016:84 , srv1454714660:84 , srv1460572290:84 , srv1476025229:84 , srv1494257574:84 , srv1519530387:84 , srv152990624:84 , srv1563103764:84 , srv1604638599:84 , srv1650165484:84 , srv1729681483:84 , srv1793847803:84 , srv1807153098:84 , srv1821499213:84 , srv1835953995:84 , srv184132068:84 , srv1848558857:84 , srv1851549305:84 , srv1858639031:84 , srv1860356601:84 , srv1922334749:84 , srv1927704905:84 , srv193058961:84 , srv1932318807:84 , srv1959158892:84 , srv2036078782:84 , srv2057880047:84 , srv2064717709:84 , srv2089297392:84 , srv2099821413:84 , srv2103335090:84 , srv2110545176:84 , srv2128203316:84 , srv270477963:84 , srv27695563:84 , srv285628241:84 , srv287454239:84 , srv298081227:84 , srv306992912:84 , srv315335259:84 , srv316723803:84 , srv328558629:84 , srv365438615:84 , srv377024829:84 , srv399394343:84 , srv404372098:84 , srv408195864:84 , srv426668662:84 , srv434731979:84 , srv440084320:84 , srv48825949:84 , srv537224307:84 , srv541517670:84 , srv563240305:84 , srv581809081:84 , srv680110246:84 , srv680782433:84 , srv740443453:84 , srv743457383:84 , srv75428322:84 , srv76456430:84 , srv787593634:84 , srv795086023:84 , srv825893304:84 , srv865243353:84 , srv868624359:84 , srv880604320:84 , srv888247810:84 , srv889891769:84 , srv898524622:84 , srv909433237:84 , srv942509609:84 , srv953257215:84 , srv954032013:84 , srv95894351:84 , srv962147114:84 , srv962847505:84 , srv1846235749:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-04 17:55:27,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:27,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv306992912=61, srv865243353=87, srv954032013=96, srv2064717709=50, srv740443453=80, srv787593634=84, srv1454714660=23, srv270477963=56, srv880604320=89, srv1432388016=22, srv1404506237=21, srv1460572290=24, srv1035555615=1, srv898524622=92, srv399394343=67, srv1350508274=19, srv1519530387=27, srv1729681483=32, srv1927704905=44, srv1821499213=35, srv193058961=45, srv1846235749=38, srv1851549305=40, srv962147114=98, srv408195864=69, srv1793847803=33, srv1807153098=34, srv1250394368=11, srv2057880047=49, srv2099821413=52, srv1304637531=16, srv1563103764=29, srv1116641820=5, srv2103335090=53, srv1244991265=10, srv298081227=60, srv2036078782=48, srv1111890536=4, srv434731979=71, srv1835953995=36, srv287454239=59, srv1394920619=20, srv1650165484=31, srv1959158892=47, srv315335259=62, srv426668662=70, srv152990624=28, srv377024829=66, srv1329406337=17, srv1181182758=8, srv316723803=63, srv95894351=97, srv888247810=90, srv942509609=94, srv1286449499=14, srv962847505=99, srv889891769=91, srv1336356916=18, srv1604638599=30, srv2110545176=54, srv1476025229=25, srv184132068=37, srv129009956=15, srv76456430=83, srv1185212772=9, srv1860356601=42, srv1017306779=0, srv1494257574=26, srv795086023=85, srv328558629=64, srv537224307=74, srv909433237=93, srv1932318807=46, srv743457383=81, srv1148584163=7, srv113448828=6, srv1858639031=41, srv285628241=58, srv2128203316=55, srv1272779064=13, srv1922334749=43, srv953257215=95, srv868624359=88, srv2089297392=51, srv27695563=57, srv365438615=65, srv75428322=82, srv581809081=77, srv1251535175=12, srv541517670=75, srv1848558857=39, srv48825949=73, srv1038325126=2, srv404372098=68, srv563240305=76, srv1087605673=3, srv825893304=86, srv680782433=79, srv680110246=78, srv440084320=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:27,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:27,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:27,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-04 17:55:27,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-04 17:55:27,534 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-04 17:55:27,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-04 17:55:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-04 17:55:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-04 17:55:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-04 17:55:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-04 17:55:27,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:27,806 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-04 17:55:27,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-04 17:55:28,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 615 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17963046820586862. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03894896754186981, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054997194092287024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03894896754186981, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:28,188 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1404506237:82 , srv1835953995:82 , srv193058961:82 , srv2036078782:82 , srv2057880047:82 , srv315335259:82 , srv743457383:82 , srv2110545176:83 , srv2128203316:83 , srv581809081:83 , srv75428322:83 , srv76456430:83 , srv1017306779:84 , srv1251535175:84 , srv1476025229:84 , srv1563103764:84 , srv1821499213:84 , srv184132068:84 , srv328558629:84 , srv541517670:84 , srv740443453:84 , srv868624359:84 , srv1035555615:85 , srv1336356916:85 , srv1454714660:85 , srv152990624:85 , srv1650165484:85 , srv1729681483:85 , srv2099821413:85 , srv270477963:85 , srv399394343:85 , srv404372098:85 , srv48825949:85 , srv680782433:85 , srv795086023:85 , srv880604320:85 , srv888247810:85 , srv889891769:85 , srv942509609:85 , srv1087605673:86 , srv1181182758:86 , srv1185212772:86 , srv1244991265:86 , srv1272779064:86 , srv129009956:86 , srv1329406337:86 , srv1460572290:86 , srv1519530387:86 , srv1793847803:86 , srv1927704905:86 , srv285628241:86 , srv287454239:86 , srv898524622:86 , srv1111890536:87 , srv1250394368:87 , srv1860356601:87 , srv2064717709:87 , srv298081227:87 , srv316723803:87 , srv365438615:87 , srv377024829:87 , srv408195864:87 , srv426668662:87 , srv787593634:87 , srv953257215:87 , srv1116641820:88 , srv1304637531:88 , srv1394920619:88 , srv1494257574:88 , srv1807153098:88 , srv1848558857:88 , srv2089297392:88 , srv27695563:88 , srv306992912:88 , srv440084320:88 , srv563240305:88 , srv865243353:88 , srv954032013:88 , srv95894351:88 , srv113448828:89 , srv1286449499:89 , srv1432388016:89 , srv1858639031:89 , srv2103335090:89 , srv434731979:89 , srv825893304:89 , srv962147114:89 , srv1148584163:90 , srv1350508274:90 , srv1851549305:90 , srv909433237:90 , srv1038325126:91 , srv1922334749:91 , srv680110246:91 , srv1932318807:92 , srv1959158892:92 , srv962847505:92 , srv537224307:93 , srv1604638599:96 , srv1846235749:436 }
2023-11-04 17:55:28,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:28,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv306992912=61, srv865243353=87, srv954032013=96, srv2064717709=50, srv740443453=80, srv787593634=84, srv1454714660=23, srv270477963=56, srv880604320=89, srv1432388016=22, srv1404506237=21, srv1460572290=24, srv1035555615=1, srv898524622=92, srv399394343=67, srv1350508274=19, srv1519530387=27, srv1729681483=32, srv1927704905=44, srv1821499213=35, srv193058961=45, srv1846235749=38, srv1851549305=40, srv962147114=98, srv408195864=69, srv1793847803=33, srv1807153098=34, srv1250394368=11, srv2057880047=49, srv2099821413=52, srv1304637531=16, srv1563103764=29, srv1116641820=5, srv2103335090=53, srv1244991265=10, srv298081227=60, srv2036078782=48, srv1111890536=4, srv434731979=71, srv1835953995=36, srv287454239=59, srv1394920619=20, srv1650165484=31, srv1959158892=47, srv315335259=62, srv426668662=70, srv152990624=28, srv377024829=66, srv1329406337=17, srv1181182758=8, srv316723803=63, srv95894351=97, srv888247810=90, srv942509609=94, srv1286449499=14, srv962847505=99, srv889891769=91, srv1336356916=18, srv1604638599=30, srv2110545176=54, srv1476025229=25, srv184132068=37, srv129009956=15, srv76456430=83, srv1185212772=9, srv1860356601=42, srv1017306779=0, srv1494257574=26, srv795086023=85, srv328558629=64, srv537224307=74, srv909433237=93, srv1932318807=46, srv743457383=81, srv1148584163=7, srv113448828=6, srv1858639031=41, srv285628241=58, srv2128203316=55, srv1272779064=13, srv1922334749=43, srv953257215=95, srv868624359=88, srv2089297392=51, srv27695563=57, srv365438615=65, srv75428322=82, srv581809081=77, srv1251535175=12, srv541517670=75, srv1848558857=39, srv48825949=73, srv1038325126=2, srv404372098=68, srv563240305=76, srv1087605673=3, srv825893304=86, srv680782433=79, srv680110246=78, srv440084320=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:28,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:28,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:28,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:28,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:28,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:28,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-04 17:55:28,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-04 17:55:28,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-04 17:55:28,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-04 17:55:28,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-04 17:55:28,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-04 17:55:28,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-04 17:55:28,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-04 17:55:28,388 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03894896754186981, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054997194092287024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03894896754186981, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:28,388 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-04 17:55:28,388 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1795976631347118, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03894896754186981, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054997194092287024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03894896754186981, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17541666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-04 17:55:28,657 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 405 ms to try 5000 different iterations.  Found a solution that moves 2606 regions; Going from a computed imbalance of 0.1795976631347118 to a new imbalance of 0.01476381038177717. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021258567076566953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034705753670287456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28955555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021258567076566953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:28,668 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1604638599:75 , srv1922334749:79 , srv537224307:79 , srv1148584163:81 , srv1350508274:81 , srv1932318807:81 , srv962847505:81 , srv1038325126:82 , srv1286449499:82 , srv1304637531:82 , srv1807153098:82 , srv1851549305:82 , srv434731979:82 , srv909433237:82 , srv1116641820:83 , srv1272779064:83 , srv1394920619:83 , srv1959158892:83 , srv2089297392:83 , srv306992912:83 , srv377024829:83 , srv680110246:83 , srv825893304:83 , srv962147114:83 , srv1432388016:84 , srv1494257574:84 , srv1793847803:84 , srv27695563:84 , srv440084320:84 , srv898524622:84 , srv1111890536:85 , srv113448828:85 , srv1181182758:85 , srv1185212772:85 , srv1244991265:85 , srv1329406337:85 , srv1563103764:85 , srv1858639031:85 , srv1860356601:85 , srv2103335090:85 , srv298081227:85 , srv365438615:85 , srv426668662:85 , srv563240305:85 , srv787593634:85 , srv953257215:85 , srv954032013:85 , srv1087605673:86 , srv1250394368:86 , srv2064717709:86 , srv2099821413:86 , srv287454239:86 , srv404372098:86 , srv408195864:86 , srv865243353:86 , srv95894351:86 , srv1035555615:87 , srv1251535175:87 , srv129009956:87 , srv1336356916:87 , srv1454714660:87 , srv1476025229:87 , srv1519530387:87 , srv1848558857:87 , srv285628241:87 , srv316723803:87 , srv399394343:87 , srv795086023:87 , srv888247810:87 , srv942509609:87 , srv1017306779:88 , srv1404506237:88 , srv1460572290:88 , srv152990624:88 , srv1650165484:88 , srv1729681483:88 , srv184132068:88 , srv1927704905:88 , srv2057880047:88 , srv270477963:88 , srv315335259:88 , srv328558629:88 , srv48825949:88 , srv541517670:88 , srv740443453:88 , srv868624359:88 , srv880604320:88 , srv889891769:88 , srv1821499213:89 , srv2110545176:89 , srv2128203316:89 , srv680782433:89 , srv75428322:89 , srv76456430:89 , srv193058961:90 , srv2036078782:90 , srv581809081:90 , srv743457383:90 , srv1835953995:93 , srv1846235749:527 }
2023-11-04 17:55:28,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:28,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv306992912=61, srv865243353=87, srv954032013=96, srv2064717709=50, srv740443453=80, srv787593634=84, srv1454714660=23, srv270477963=56, srv880604320=89, srv1432388016=22, srv1404506237=21, srv1460572290=24, srv1035555615=1, srv898524622=92, srv399394343=67, srv1350508274=19, srv1519530387=27, srv1729681483=32, srv1927704905=44, srv1821499213=35, srv193058961=45, srv1846235749=38, srv1851549305=40, srv962147114=98, srv408195864=69, srv1793847803=33, srv1807153098=34, srv1250394368=11, srv2057880047=49, srv2099821413=52, srv1304637531=16, srv1563103764=29, srv1116641820=5, srv2103335090=53, srv1244991265=10, srv298081227=60, srv2036078782=48, srv1111890536=4, srv434731979=71, srv1835953995=36, srv287454239=59, srv1394920619=20, srv1650165484=31, srv1959158892=47, srv315335259=62, srv426668662=70, srv152990624=28, srv377024829=66, srv1329406337=17, srv1181182758=8, srv316723803=63, srv95894351=97, srv888247810=90, srv942509609=94, srv1286449499=14, srv962847505=99, srv889891769=91, srv1336356916=18, srv1604638599=30, srv2110545176=54, srv1476025229=25, srv184132068=37, srv129009956=15, srv76456430=83, srv1185212772=9, srv1860356601=42, srv1017306779=0, srv1494257574=26, srv795086023=85, srv328558629=64, srv537224307=74, srv909433237=93, srv1932318807=46, srv743457383=81, srv1148584163=7, srv113448828=6, srv1858639031=41, srv285628241=58, srv2128203316=55, srv1272779064=13, srv1922334749=43, srv953257215=95, srv868624359=88, srv2089297392=51, srv27695563=57, srv365438615=65, srv75428322=82, srv581809081=77, srv1251535175=12, srv541517670=75, srv1848558857=39, srv48825949=73, srv1038325126=2, srv404372098=68, srv563240305=76, srv1087605673=3, srv825893304=86, srv680782433=79, srv680110246=78, srv440084320=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:28,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:28,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:28,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:28,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-04 17:55:28,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-04 17:55:28,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-04 17:55:28,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-04 17:55:28,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-04 17:55:28,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-04 17:55:28,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-04 17:55:28,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-04 17:55:28,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-04 17:55:28,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-04 17:55:28,831 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021258567076566953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034705753670287456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021258567076566953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:28,831 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-04 17:55:28,831 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014745562779763351, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021258567076566953); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034705753670287456, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021258567076566953); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-04 17:55:29,077 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 348 ms to try 5000 different iterations.  Found a solution that moves 975 regions; Going from a computed imbalance of 0.014745562779763351 to a new imbalance of 4.5481814461656905E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004436018949768876); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008109796489563252); MoveCostFunction : (multiplier=7.0, imbalance=0.10833333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004436018949768876); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:29,080 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1848558857:82 , srv113448828:83 , srv1959158892:83 , srv680110246:83 , srv1038325126:84 , srv129009956:84 , srv1432388016:84 , srv1460572290:84 , srv1519530387:84 , srv1729681483:84 , srv1835953995:84 , srv1858639031:84 , srv1927704905:84 , srv1932318807:84 , srv2103335090:84 , srv285628241:84 , srv316723803:84 , srv48825949:84 , srv680782433:84 , srv865243353:84 , srv880604320:84 , srv962847505:84 , srv1017306779:85 , srv1250394368:85 , srv1454714660:85 , srv1494257574:85 , srv152990624:85 , srv1650165484:85 , srv184132068:85 , srv193058961:85 , srv2036078782:85 , srv2064717709:85 , srv2110545176:85 , srv2128203316:85 , srv328558629:85 , srv399394343:85 , srv408195864:85 , srv440084320:85 , srv563240305:85 , srv743457383:85 , srv76456430:85 , srv795086023:85 , srv825893304:85 , srv888247810:85 , srv889891769:85 , srv909433237:85 , srv942509609:85 , srv954032013:85 , srv95894351:85 , srv1035555615:86 , srv1087605673:86 , srv1111890536:86 , srv1116641820:86 , srv1148584163:86 , srv1181182758:86 , srv1244991265:86 , srv1329406337:86 , srv1394920619:86 , srv1476025229:86 , srv1604638599:86 , srv1821499213:86 , srv1851549305:86 , srv1860356601:86 , srv2099821413:86 , srv270477963:86 , srv27695563:86 , srv287454239:86 , srv298081227:86 , srv365438615:86 , srv404372098:86 , srv426668662:86 , srv434731979:86 , srv537224307:86 , srv541517670:86 , srv581809081:86 , srv740443453:86 , srv75428322:86 , srv787593634:86 , srv868624359:86 , srv953257215:86 , srv1185212772:87 , srv1251535175:87 , srv1286449499:87 , srv1304637531:87 , srv1336356916:87 , srv1350508274:87 , srv1807153098:87 , srv1922334749:87 , srv2089297392:87 , srv315335259:87 , srv898524622:87 , srv962147114:87 , srv1272779064:88 , srv1404506237:88 , srv1793847803:88 , srv2057880047:88 , srv306992912:88 , srv377024829:88 , srv1563103764:90 , srv1846235749:534 }
2023-11-04 17:55:29,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:29,084 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv306992912=61, srv865243353=87, srv954032013=96, srv2064717709=50, srv740443453=80, srv787593634=84, srv1454714660=23, srv270477963=56, srv880604320=89, srv1432388016=22, srv1404506237=21, srv1460572290=24, srv1035555615=1, srv898524622=92, srv399394343=67, srv1350508274=19, srv1519530387=27, srv1729681483=32, srv1927704905=44, srv1821499213=35, srv193058961=45, srv1846235749=38, srv1851549305=40, srv962147114=98, srv408195864=69, srv1793847803=33, srv1807153098=34, srv1250394368=11, srv2057880047=49, srv2099821413=52, srv1304637531=16, srv1563103764=29, srv1116641820=5, srv2103335090=53, srv1244991265=10, srv298081227=60, srv2036078782=48, srv1111890536=4, srv434731979=71, srv1835953995=36, srv287454239=59, srv1394920619=20, srv1650165484=31, srv1959158892=47, srv315335259=62, srv426668662=70, srv152990624=28, srv377024829=66, srv1329406337=17, srv1181182758=8, srv316723803=63, srv95894351=97, srv888247810=90, srv942509609=94, srv1286449499=14, srv962847505=99, srv889891769=91, srv1336356916=18, srv1604638599=30, srv2110545176=54, srv1476025229=25, srv184132068=37, srv129009956=15, srv76456430=83, srv1185212772=9, srv1860356601=42, srv1017306779=0, srv1494257574=26, srv795086023=85, srv328558629=64, srv537224307=74, srv909433237=93, srv1932318807=46, srv743457383=81, srv1148584163=7, srv113448828=6, srv1858639031=41, srv285628241=58, srv2128203316=55, srv1272779064=13, srv1922334749=43, srv953257215=95, srv868624359=88, srv2089297392=51, srv27695563=57, srv365438615=65, srv75428322=82, srv581809081=77, srv1251535175=12, srv541517670=75, srv1848558857=39, srv48825949=73, srv1038325126=2, srv404372098=68, srv563240305=76, srv1087605673=3, srv825893304=86, srv680782433=79, srv680110246=78, srv440084320=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:29,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:29,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:29,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-04 17:55:29,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-04 17:55:29,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-04 17:55:29,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-04 17:55:29,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-04 17:55:29,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-04 17:55:29,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-04 17:55:29,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-04 17:55:29,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-04 17:55:29,217 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004436018949768876); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008109796489563252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004436018949768876); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:29,217 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-04 17:55:29,218 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.4799104869812215E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004436018949768876); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008109796489563252); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004436018949768876); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-04 17:55:29,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 4.4799104869812215E-4 to a new imbalance of 1.9916356746200808E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:29,576 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1336356916:83 , srv1563103764:83 , srv1821499213:83 , srv1835953995:83 , srv1860356601:83 , srv2110545176:83 , srv270477963:83 , srv95894351:83 , srv962147114:83 , srv1017306779:84 , srv1035555615:84 , srv1087605673:84 , srv1111890536:84 , srv1185212772:84 , srv1251535175:84 , srv1286449499:84 , srv1350508274:84 , srv1404506237:84 , srv152990624:84 , srv1650165484:84 , srv1793847803:84 , srv1807153098:84 , srv1848558857:84 , srv1851549305:84 , srv1858639031:84 , srv1959158892:84 , srv2057880047:84 , srv2064717709:84 , srv2089297392:84 , srv2103335090:84 , srv27695563:84 , srv285628241:84 , srv287454239:84 , srv298081227:84 , srv306992912:84 , srv315335259:84 , srv365438615:84 , srv377024829:84 , srv408195864:84 , srv426668662:84 , srv541517670:84 , srv563240305:84 , srv581809081:84 , srv680782433:84 , srv740443453:84 , srv75428322:84 , srv787593634:84 , srv865243353:84 , srv868624359:84 , srv889891769:84 , srv942509609:84 , srv953257215:84 , srv954032013:84 , srv1038325126:85 , srv1116641820:85 , srv113448828:85 , srv1148584163:85 , srv1181182758:85 , srv1244991265:85 , srv1250394368:85 , srv1272779064:85 , srv129009956:85 , srv1304637531:85 , srv1329406337:85 , srv1394920619:85 , srv1432388016:85 , srv1454714660:85 , srv1460572290:85 , srv1476025229:85 , srv1494257574:85 , srv1519530387:85 , srv1604638599:85 , srv1729681483:85 , srv1927704905:85 , srv193058961:85 , srv1932318807:85 , srv2036078782:85 , srv2099821413:85 , srv2128203316:85 , srv316723803:85 , srv328558629:85 , srv399394343:85 , srv404372098:85 , srv434731979:85 , srv440084320:85 , srv48825949:85 , srv537224307:85 , srv680110246:85 , srv743457383:85 , srv76456430:85 , srv795086023:85 , srv825893304:85 , srv880604320:85 , srv888247810:85 , srv898524622:85 , srv962847505:85 , srv184132068:86 , srv1922334749:86 , srv909433237:86 , srv1846235749:644 }
2023-11-04 17:55:29,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:29,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv306992912=61, srv865243353=87, srv954032013=96, srv2064717709=50, srv740443453=80, srv787593634=84, srv1454714660=23, srv270477963=56, srv880604320=89, srv1432388016=22, srv1404506237=21, srv1460572290=24, srv1035555615=1, srv898524622=92, srv399394343=67, srv1350508274=19, srv1519530387=27, srv1729681483=32, srv1927704905=44, srv1821499213=35, srv193058961=45, srv1846235749=38, srv1851549305=40, srv962147114=98, srv408195864=69, srv1793847803=33, srv1807153098=34, srv1250394368=11, srv2057880047=49, srv2099821413=52, srv1304637531=16, srv1563103764=29, srv1116641820=5, srv2103335090=53, srv1244991265=10, srv298081227=60, srv2036078782=48, srv1111890536=4, srv434731979=71, srv1835953995=36, srv287454239=59, srv1394920619=20, srv1650165484=31, srv1959158892=47, srv315335259=62, srv426668662=70, srv152990624=28, srv377024829=66, srv1329406337=17, srv1181182758=8, srv316723803=63, srv95894351=97, srv888247810=90, srv942509609=94, srv1286449499=14, srv962847505=99, srv889891769=91, srv1336356916=18, srv1604638599=30, srv2110545176=54, srv1476025229=25, srv184132068=37, srv129009956=15, srv76456430=83, srv1185212772=9, srv1860356601=42, srv1017306779=0, srv1494257574=26, srv795086023=85, srv328558629=64, srv537224307=74, srv909433237=93, srv1932318807=46, srv743457383=81, srv1148584163=7, srv113448828=6, srv1858639031=41, srv285628241=58, srv2128203316=55, srv1272779064=13, srv1922334749=43, srv953257215=95, srv868624359=88, srv2089297392=51, srv27695563=57, srv365438615=65, srv75428322=82, srv581809081=77, srv1251535175=12, srv541517670=75, srv1848558857=39, srv48825949=73, srv1038325126=2, srv404372098=68, srv563240305=76, srv1087605673=3, srv825893304=86, srv680782433=79, srv680110246=78, srv440084320=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-04 17:55:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-04 17:55:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-04 17:55:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-04 17:55:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-04 17:55:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-04 17:55:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-04 17:55:29,714 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8284855875433483E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.23782800878924E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.23782800878924E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:29,714 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1336356916:83 , srv1563103764:83 , srv1821499213:83 , srv1835953995:83 , srv1860356601:83 , srv2110545176:83 , srv270477963:83 , srv95894351:83 , srv962147114:83 , srv1017306779:84 , srv1035555615:84 , srv1087605673:84 , srv1111890536:84 , srv1185212772:84 , srv1251535175:84 , srv1286449499:84 , srv1350508274:84 , srv1404506237:84 , srv152990624:84 , srv1650165484:84 , srv1793847803:84 , srv1807153098:84 , srv1848558857:84 , srv1851549305:84 , srv1858639031:84 , srv1959158892:84 , srv2057880047:84 , srv2064717709:84 , srv2089297392:84 , srv2103335090:84 , srv27695563:84 , srv285628241:84 , srv287454239:84 , srv298081227:84 , srv306992912:84 , srv315335259:84 , srv365438615:84 , srv377024829:84 , srv408195864:84 , srv426668662:84 , srv541517670:84 , srv563240305:84 , srv581809081:84 , srv680782433:84 , srv740443453:84 , srv75428322:84 , srv787593634:84 , srv865243353:84 , srv868624359:84 , srv889891769:84 , srv942509609:84 , srv953257215:84 , srv954032013:84 , srv1038325126:85 , srv1116641820:85 , srv113448828:85 , srv1148584163:85 , srv1181182758:85 , srv1244991265:85 , srv1250394368:85 , srv1272779064:85 , srv129009956:85 , srv1304637531:85 , srv1329406337:85 , srv1394920619:85 , srv1432388016:85 , srv1454714660:85 , srv1460572290:85 , srv1476025229:85 , srv1494257574:85 , srv1519530387:85 , srv1604638599:85 , srv1729681483:85 , srv1927704905:85 , srv193058961:85 , srv1932318807:85 , srv2036078782:85 , srv2099821413:85 , srv2128203316:85 , srv316723803:85 , srv328558629:85 , srv399394343:85 , srv404372098:85 , srv434731979:85 , srv440084320:85 , srv48825949:85 , srv537224307:85 , srv680110246:85 , srv743457383:85 , srv76456430:85 , srv795086023:85 , srv825893304:85 , srv880604320:85 , srv888247810:85 , srv898524622:85 , srv962847505:85 , srv184132068:86 , srv1922334749:86 , srv909433237:86 , srv1846235749:644 }
2023-11-04 17:55:29,750 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=166 (was 146) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=6037 (was 6527)
2023-11-04 17:55:29,765 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=166, ProcessCount=170, AvailableMemoryMB=6036
2023-11-04 17:55:29,767 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-04 17:55:29,767 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-04 17:55:29,779 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1145478656:84 , srv1298344172:84 , srv1361788933:84 , srv1414968372:84 , srv1457340568:84 , srv1460690981:84 , srv1488878798:84 , srv1545986877:84 , srv1553348423:84 , srv1570400062:84 , srv1604644438:84 , srv1607601102:84 , srv1609457523:84 , srv1641601905:84 , srv1785622929:84 , srv1824744935:84 , srv1826595549:84 , srv1947672732:84 , srv195032010:84 , srv1957740227:84 , srv2064143565:84 , srv227804806:84 , srv263870296:84 , srv336703881:84 , srv531661758:84 , srv723309975:84 , srv807784051:84 , srv873680934:84 , srv890061111:84 , srv1350525413:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-04 17:55:29,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:55:29,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv890061111=29, srv1457340568=5, srv1553348423=9, srv531661758=25, srv1826595549=17, srv1607601102=12, srv1641601905=14, srv807784051=27, srv263870296=23, srv1488878798=7, srv336703881=24, srv1824744935=16, srv2064143565=21, srv1609457523=13, srv1947672732=18, srv195032010=19, srv1545986877=8, srv1785622929=15, srv723309975=26, srv873680934=28, srv1570400062=10, srv1145478656=0, srv1350525413=2, srv1604644438=11, srv1957740227=20, srv1460690981=6, srv227804806=22, srv1298344172=1, srv1361788933=3, srv1414968372=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:55:29,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:55:29,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:55:29,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-04 17:55:29,817 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1331967799804474, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:55:29,818 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912612741968723, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1331967799804474, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-04 17:57:29,801 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5482836 different iterations.  Found a solution that moves 1877 regions; Going from a computed imbalance of 0.9912612741968723 to a new imbalance of 4.3810116372393E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6951851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:57:29,807 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1145478656:90 , srv1298344172:90 , srv1350525413:90 , srv1361788933:90 , srv1414968372:90 , srv1457340568:90 , srv1460690981:90 , srv1488878798:90 , srv1545986877:90 , srv1553348423:90 , srv1570400062:90 , srv1604644438:90 , srv1607601102:90 , srv1609457523:90 , srv1641601905:90 , srv1785622929:90 , srv1824744935:90 , srv1826595549:90 , srv1947672732:90 , srv195032010:90 , srv1957740227:90 , srv2064143565:90 , srv227804806:90 , srv263870296:90 , srv336703881:90 , srv531661758:90 , srv723309975:90 , srv807784051:90 , srv873680934:90 , srv890061111:90 }
2023-11-04 17:57:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-04 17:57:29,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv890061111=29, srv1457340568=5, srv1553348423=9, srv531661758=25, srv1826595549=17, srv1607601102=12, srv1641601905=14, srv807784051=27, srv263870296=23, srv1488878798=7, srv336703881=24, srv1824744935=16, srv2064143565=21, srv1609457523=13, srv1947672732=18, srv195032010=19, srv1545986877=8, srv1785622929=15, srv723309975=26, srv873680934=28, srv1570400062=10, srv1145478656=0, srv1350525413=2, srv1604644438=11, srv1957740227=20, srv1460690981=6, srv227804806=22, srv1298344172=1, srv1361788933=3, srv1414968372=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-04 17:57:29,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-04 17:57:29,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-04 17:57:29,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-04 17:57:29,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-04 17:57:29,825 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-04 17:57:29,825 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1145478656:90 , srv1298344172:90 , srv1350525413:90 , srv1361788933:90 , srv1414968372:90 , srv1457340568:90 , srv1460690981:90 , srv1488878798:90 , srv1545986877:90 , srv1553348423:90 , srv1570400062:90 , srv1604644438:90 , srv1607601102:90 , srv1609457523:90 , srv1641601905:90 , srv1785622929:90 , srv1824744935:90 , srv1826595549:90 , srv1947672732:90 , srv195032010:90 , srv1957740227:90 , srv2064143565:90 , srv227804806:90 , srv263870296:90 , srv336703881:90 , srv531661758:90 , srv723309975:90 , srv807784051:90 , srv873680934:90 , srv890061111:90 }
2023-11-04 17:57:29,837 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=258 (was 166) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 170), AvailableMemoryMB=5981 (was 6036)