2023-11-11 06:55:11,447 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-11 06:55:11,722 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-11 06:55:11,822 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-11 06:55:11,887 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 06:55:11,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-11 06:55:11,907 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=167, AvailableMemoryMB=2935
2023-11-11 06:55:11,909 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 06:55:11,909 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-11 06:55:12,038 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006349967:84 , srv1017652792:84 , srv1033582788:84 , srv1079213958:84 , srv1129688206:84 , srv1169876404:84 , srv1187139770:84 , srv1191323945:84 , srv1209279420:84 , srv1223713509:84 , srv1264097982:84 , srv1270228988:84 , srv1280986321:84 , srv1296168706:84 , srv130423643:84 , srv1360731257:84 , srv138853375:84 , srv1416352832:84 , srv145823731:84 , srv1502779866:84 , srv1527216169:84 , srv1542500388:84 , srv1559951217:84 , srv1606075359:84 , srv1607230784:84 , srv1643150158:84 , srv1654072595:84 , srv1656858783:84 , srv1657529043:84 , srv1659104754:84 , srv166630449:84 , srv1686853594:84 , srv1688583344:84 , srv1721314252:84 , srv1813990417:84 , srv1814534105:84 , srv1827821546:84 , srv1841480762:84 , srv1847004344:84 , srv1847509338:84 , srv1885644285:84 , srv1909213339:84 , srv1954371100:84 , srv196674924:84 , srv1994031907:84 , srv2001210251:84 , srv2027067909:84 , srv2040534305:84 , srv2042434512:84 , srv2053951669:84 , srv2067126166:84 , srv2125338194:84 , srv2125511565:84 , srv2138409723:84 , srv260330674:84 , srv26109707:84 , srv2652171:84 , srv270638664:84 , srv280237146:84 , srv281407866:84 , srv282274588:84 , srv318460076:84 , srv323650811:84 , srv335194478:84 , srv353138507:84 , srv356328359:84 , srv356796626:84 , srv377436115:84 , srv397241486:84 , srv402138550:84 , srv404882606:84 , srv429291410:84 , srv436484516:84 , srv442078874:84 , srv497295421:84 , srv502588795:84 , srv50315031:84 , srv530998855:84 , srv549886739:84 , srv564832497:84 , srv5757510:84 , srv607407385:84 , srv612701463:84 , srv654344976:84 , srv675101007:84 , srv715081620:84 , srv745232943:84 , srv780759362:84 , srv806532160:84 , srv833787545:84 , srv836739667:84 , srv844628717:84 , srv86798776:84 , srv871291372:84 , srv874334184:84 , srv89622799:84 , srv949126009:84 , srv966201220:84 , srv995194179:84 , srv109969222:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-11 06:55:12,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:12,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:12,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:12,127 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:12,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:12,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:12,130 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:12,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:12,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:12,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:12,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:12,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:12,333 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:12,334 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 06:55:12,334 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 06:55:12,779 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 618 ms to try 5000 different iterations.  Found a solution that moves 4701 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17804367930939388. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041938916025467274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0575948931518387, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5223333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041938916025467274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:12,796 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1129688206:82 , srv1606075359:82 , srv2053951669:82 , srv1169876404:83 , srv166630449:83 , srv1813990417:83 , srv282274588:83 , srv1079213958:84 , srv1209279420:84 , srv1280986321:84 , srv1360731257:84 , srv1416352832:84 , srv1527216169:84 , srv1559951217:84 , srv1686853594:84 , srv2067126166:84 , srv270638664:84 , srv497295421:84 , srv564832497:84 , srv5757510:84 , srv806532160:84 , srv871291372:84 , srv1033582788:85 , srv1191323945:85 , srv1223713509:85 , srv130423643:85 , srv1607230784:85 , srv1654072595:85 , srv1657529043:85 , srv196674924:85 , srv1994031907:85 , srv2001210251:85 , srv2042434512:85 , srv2125511565:85 , srv2652171:85 , srv323650811:85 , srv356796626:85 , srv436484516:85 , srv442078874:85 , srv549886739:85 , srv745232943:85 , srv836739667:85 , srv949126009:85 , srv995194179:85 , srv1296168706:86 , srv145823731:86 , srv1542500388:86 , srv1688583344:86 , srv2138409723:86 , srv260330674:86 , srv26109707:86 , srv280237146:86 , srv356328359:86 , srv402138550:86 , srv404882606:86 , srv502588795:86 , srv50315031:86 , srv780759362:86 , srv844628717:86 , srv1017652792:87 , srv1187139770:87 , srv1270228988:87 , srv1502779866:87 , srv1841480762:87 , srv1885644285:87 , srv1909213339:87 , srv2125338194:87 , srv429291410:87 , srv530998855:87 , srv607407385:87 , srv654344976:87 , srv675101007:87 , srv833787545:87 , srv86798776:87 , srv874334184:87 , srv1006349967:88 , srv1643150158:88 , srv1656858783:88 , srv1721314252:88 , srv2027067909:88 , srv318460076:88 , srv715081620:88 , srv1264097982:89 , srv138853375:89 , srv1847509338:89 , srv1954371100:89 , srv353138507:89 , srv377436115:89 , srv612701463:89 , srv1659104754:90 , srv1814534105:90 , srv1827821546:90 , srv1847004344:90 , srv281407866:90 , srv89622799:90 , srv966201220:91 , srv2040534305:92 , srv397241486:92 , srv335194478:93 , srv109969222:463 }
2023-11-11 06:55:12,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:12,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:12,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:12,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:12,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:12,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:12,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:12,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:12,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:12,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:12,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041938916025467274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0575948931518387, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041938916025467274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:12,976 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 06:55:12,976 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1780107622038425, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041938916025467274, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0575948931518387, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041938916025467274, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17316666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 06:55:13,239 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 392 ms to try 5000 different iterations.  Found a solution that moves 2593 regions; Going from a computed imbalance of 0.1780107622038425 to a new imbalance of 0.01352603169176469. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023948650225261582); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03520343315570759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2881111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023948650225261582); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:13,248 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv335194478:78 , srv397241486:79 , srv1847004344:80 , srv2040534305:80 , srv281407866:81 , srv1006349967:82 , srv1814534105:82 , srv1827821546:82 , srv353138507:82 , srv612701463:82 , srv715081620:82 , srv89622799:82 , srv1264097982:83 , srv138853375:83 , srv1656858783:83 , srv1954371100:83 , srv1296168706:84 , srv1502779866:84 , srv1643150158:84 , srv1659104754:84 , srv1721314252:84 , srv1847509338:84 , srv1909213339:84 , srv2138409723:84 , srv280237146:84 , srv377436115:84 , srv429291410:84 , srv654344976:84 , srv780759362:84 , srv1017652792:85 , srv1187139770:85 , srv1191323945:85 , srv1270228988:85 , srv1688583344:85 , srv2027067909:85 , srv2125338194:85 , srv2125511565:85 , srv26109707:85 , srv323650811:85 , srv356328359:85 , srv404882606:85 , srv436484516:85 , srv675101007:85 , srv86798776:85 , srv874334184:85 , srv949126009:85 , srv966201220:85 , srv1360731257:86 , srv145823731:86 , srv1542500388:86 , srv1607230784:86 , srv1885644285:86 , srv1994031907:86 , srv2042434512:86 , srv260330674:86 , srv502588795:86 , srv50315031:86 , srv549886739:86 , srv833787545:86 , srv836739667:86 , srv995194179:86 , srv1033582788:87 , srv1223713509:87 , srv1280986321:87 , srv1657529043:87 , srv166630449:87 , srv1841480762:87 , srv196674924:87 , srv2001210251:87 , srv270638664:87 , srv402138550:87 , srv442078874:87 , srv497295421:87 , srv530998855:87 , srv5757510:87 , srv607407385:87 , srv745232943:87 , srv844628717:87 , srv871291372:87 , srv1079213958:88 , srv1209279420:88 , srv130423643:88 , srv1559951217:88 , srv2067126166:88 , srv318460076:88 , srv564832497:88 , srv1129688206:89 , srv1169876404:89 , srv1416352832:89 , srv1527216169:89 , srv1654072595:89 , srv1686853594:89 , srv282274588:89 , srv356796626:89 , srv2053951669:90 , srv2652171:90 , srv806532160:90 , srv1606075359:91 , srv1813990417:91 , srv109969222:524 }
2023-11-11 06:55:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:13,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:13,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:13,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:13,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:13,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:13,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:13,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:13,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:13,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:13,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:13,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:13,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:13,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:13,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:13,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:13,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:13,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:13,405 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023948650225261582); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03520343315570759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023948650225261582); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:13,405 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 06:55:13,405 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01350787511769645, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023948650225261582); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03520343315570759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023948650225261582); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042833333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 06:55:13,654 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 348 ms to try 5000 different iterations.  Found a solution that moves 952 regions; Going from a computed imbalance of 0.01350787511769645 to a new imbalance of 8.546727367214323E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006203515249160302); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006750316948122676); MoveCostFunction : (multiplier=7.0, imbalance=0.10577777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006203515249160302); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:13,656 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv318460076:81 , srv966201220:82 , srv1654072595:83 , srv1841480762:83 , srv2652171:83 , srv530998855:83 , srv130423643:84 , srv1416352832:84 , srv1606075359:84 , srv1659104754:84 , srv1686853594:84 , srv356796626:84 , srv377436115:84 , srv806532160:84 , srv1017652792:85 , srv1079213958:85 , srv1169876404:85 , srv1187139770:85 , srv1209279420:85 , srv145823731:85 , srv1527216169:85 , srv1643150158:85 , srv1657529043:85 , srv1813990417:85 , srv1814534105:85 , srv1827821546:85 , srv1847509338:85 , srv1885644285:85 , srv1954371100:85 , srv196674924:85 , srv2027067909:85 , srv2053951669:85 , srv260330674:85 , srv402138550:85 , srv442078874:85 , srv50315031:85 , srv564832497:85 , srv607407385:85 , srv675101007:85 , srv833787545:85 , srv844628717:85 , srv86798776:85 , srv89622799:85 , srv1223713509:86 , srv1264097982:86 , srv1270228988:86 , srv138853375:86 , srv1502779866:86 , srv1542500388:86 , srv1559951217:86 , srv1607230784:86 , srv1656858783:86 , srv1688583344:86 , srv1721314252:86 , srv1909213339:86 , srv1994031907:86 , srv2001210251:86 , srv2040534305:86 , srv2042434512:86 , srv2067126166:86 , srv2125338194:86 , srv26109707:86 , srv270638664:86 , srv281407866:86 , srv282274588:86 , srv335194478:86 , srv353138507:86 , srv429291410:86 , srv497295421:86 , srv502588795:86 , srv5757510:86 , srv654344976:86 , srv745232943:86 , srv836739667:86 , srv874334184:86 , srv995194179:86 , srv1006349967:87 , srv1033582788:87 , srv1129688206:87 , srv1191323945:87 , srv1280986321:87 , srv1296168706:87 , srv1360731257:87 , srv166630449:87 , srv1847004344:87 , srv280237146:87 , srv323650811:87 , srv356328359:87 , srv397241486:87 , srv404882606:87 , srv436484516:87 , srv549886739:87 , srv612701463:87 , srv715081620:87 , srv871291372:87 , srv949126009:87 , srv2125511565:88 , srv2138409723:88 , srv780759362:88 , srv109969222:526 }
2023-11-11 06:55:13,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:13,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:13,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:13,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:13,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:13,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:13,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:13,783 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006203515249160302); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006750316948122676); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006203515249160302); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:13,783 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 06:55:13,783 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.480066902451642E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006203515249160302); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006750316948122676); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006203515249160302); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 06:55:14,147 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 265 regions; Going from a computed imbalance of 8.480066902451642E-4 to a new imbalance of 2.8190242406756814E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.029444444444444443, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:14,148 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1033582788:83 , srv1264097982:83 , srv1813990417:83 , srv2125338194:83 , srv1129688206:84 , srv1223713509:84 , srv1270228988:84 , srv1280986321:84 , srv1360731257:84 , srv138853375:84 , srv1527216169:84 , srv1559951217:84 , srv1656858783:84 , srv1659104754:84 , srv1721314252:84 , srv1847509338:84 , srv1885644285:84 , srv1954371100:84 , srv2001210251:84 , srv2027067909:84 , srv2040534305:84 , srv2067126166:84 , srv2125511565:84 , srv2138409723:84 , srv282274588:84 , srv335194478:84 , srv356328359:84 , srv356796626:84 , srv377436115:84 , srv397241486:84 , srv402138550:84 , srv404882606:84 , srv502588795:84 , srv549886739:84 , srv607407385:84 , srv612701463:84 , srv654344976:84 , srv745232943:84 , srv780759362:84 , srv806532160:84 , srv833787545:84 , srv844628717:84 , srv871291372:84 , srv874334184:84 , srv966201220:84 , srv1006349967:85 , srv1017652792:85 , srv1079213958:85 , srv1169876404:85 , srv1187139770:85 , srv1191323945:85 , srv1209279420:85 , srv1296168706:85 , srv130423643:85 , srv1416352832:85 , srv145823731:85 , srv1502779866:85 , srv1542500388:85 , srv1606075359:85 , srv1607230784:85 , srv1643150158:85 , srv1654072595:85 , srv1657529043:85 , srv166630449:85 , srv1686853594:85 , srv1688583344:85 , srv1814534105:85 , srv1827821546:85 , srv1841480762:85 , srv1847004344:85 , srv1909213339:85 , srv196674924:85 , srv1994031907:85 , srv2053951669:85 , srv26109707:85 , srv2652171:85 , srv270638664:85 , srv280237146:85 , srv281407866:85 , srv318460076:85 , srv323650811:85 , srv429291410:85 , srv436484516:85 , srv497295421:85 , srv50315031:85 , srv530998855:85 , srv564832497:85 , srv675101007:85 , srv715081620:85 , srv836739667:85 , srv86798776:85 , srv89622799:85 , srv995194179:85 , srv2042434512:86 , srv260330674:86 , srv353138507:86 , srv442078874:86 , srv5757510:86 , srv949126009:86 , srv109969222:628 }
2023-11-11 06:55:14,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:14,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:14,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:14,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:14,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:14,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:14,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:14,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:14,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:14,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:14,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:14,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:14,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:14,268 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:14,268 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 06:55:14,269 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6334672746871227E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 06:55:14,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 341 ms to try 5000 different iterations.  Found a solution that moves 31 regions; Going from a computed imbalance of 2.6334672746871227E-5 to a new imbalance of 6.615052311728012E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0034444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:14,517 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1542500388:83 , srv2042434512:83 , srv260330674:83 , srv2652171:83 , srv353138507:83 , srv442078874:83 , srv5757510:83 , srv607407385:83 , srv949126009:83 , srv1006349967:84 , srv1017652792:84 , srv1033582788:84 , srv1079213958:84 , srv1129688206:84 , srv1169876404:84 , srv1187139770:84 , srv1191323945:84 , srv1209279420:84 , srv1223713509:84 , srv1270228988:84 , srv1280986321:84 , srv1296168706:84 , srv130423643:84 , srv138853375:84 , srv1416352832:84 , srv145823731:84 , srv1502779866:84 , srv1527216169:84 , srv1559951217:84 , srv1606075359:84 , srv1607230784:84 , srv1643150158:84 , srv1654072595:84 , srv1657529043:84 , srv1659104754:84 , srv166630449:84 , srv1686853594:84 , srv1688583344:84 , srv1721314252:84 , srv1813990417:84 , srv1814534105:84 , srv1827821546:84 , srv1841480762:84 , srv1847004344:84 , srv1847509338:84 , srv1885644285:84 , srv1909213339:84 , srv196674924:84 , srv1994031907:84 , srv2001210251:84 , srv2027067909:84 , srv2040534305:84 , srv2053951669:84 , srv2067126166:84 , srv2125511565:84 , srv2138409723:84 , srv26109707:84 , srv270638664:84 , srv280237146:84 , srv281407866:84 , srv282274588:84 , srv318460076:84 , srv323650811:84 , srv356328359:84 , srv356796626:84 , srv397241486:84 , srv402138550:84 , srv404882606:84 , srv429291410:84 , srv436484516:84 , srv497295421:84 , srv502588795:84 , srv50315031:84 , srv530998855:84 , srv549886739:84 , srv564832497:84 , srv612701463:84 , srv675101007:84 , srv715081620:84 , srv745232943:84 , srv780759362:84 , srv806532160:84 , srv833787545:84 , srv836739667:84 , srv844628717:84 , srv86798776:84 , srv871291372:84 , srv874334184:84 , srv89622799:84 , srv966201220:84 , srv995194179:84 , srv1264097982:85 , srv1360731257:85 , srv1656858783:85 , srv1954371100:85 , srv2125338194:85 , srv335194478:85 , srv377436115:85 , srv654344976:85 , srv109969222:685 }
2023-11-11 06:55:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:14,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv402138550=70, srv353138507=65, srv1079213958=3, srv1280986321=13, srv2053951669=50, srv318460076=62, srv335194478=64, srv1686853594=32, srv2652171=57, srv166630449=31, srv138853375=17, srv836739667=91, srv1659104754=30, srv1502779866=20, srv282274588=61, srv323650811=63, srv1191323945=8, srv607407385=82, srv497295421=75, srv2067126166=51, srv1657529043=29, srv2042434512=49, srv5757510=81, srv780759362=88, srv280237146=59, srv1643150158=26, srv1885644285=41, srv844628717=92, srv1033582788=2, srv1209279420=9, srv1688583344=33, srv530998855=78, srv2125511565=53, srv1264097982=11, srv1416352832=18, srv1656858783=28, srv1721314252=34, srv442078874=74, srv397241486=69, srv429291410=72, srv874334184=95, srv130423643=15, srv502588795=76, srv715081620=86, srv1187139770=7, srv1169876404=6, srv2001210251=46, srv89622799=96, srv1909213339=42, srv260330674=55, srv612701463=83, srv109969222=4, srv404882606=71, srv1527216169=21, srv1954371100=43, srv1847509338=40, srv745232943=87, srv50315031=77, srv356796626=67, srv377436115=68, srv356328359=66, srv1994031907=45, srv1360731257=16, srv2027067909=47, srv436484516=73, srv1006349967=0, srv86798776=93, srv1813990417=35, srv1606075359=24, srv281407866=60, srv2040534305=48, srv26109707=56, srv1847004344=39, srv549886739=79, srv1017652792=1, srv1654072595=27, srv1296168706=14, srv196674924=44, srv1129688206=5, srv1841480762=38, srv1814534105=36, srv1827821546=37, srv2125338194=52, srv871291372=94, srv1542500388=22, srv654344976=84, srv675101007=85, srv564832497=80, srv1223713509=10, srv270638664=58, srv1559951217=23, srv1607230784=25, srv966201220=98, srv995194179=99, srv806532160=89, srv145823731=19, srv2138409723=54, srv1270228988=12, srv949126009=97, srv833787545=90} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 06:55:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 06:55:14,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 06:55:14,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 06:55:14,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 06:55:14,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 06:55:14,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:14,647 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1542500388:83 , srv2042434512:83 , srv260330674:83 , srv2652171:83 , srv353138507:83 , srv442078874:83 , srv5757510:83 , srv607407385:83 , srv949126009:83 , srv1006349967:84 , srv1017652792:84 , srv1033582788:84 , srv1079213958:84 , srv1129688206:84 , srv1169876404:84 , srv1187139770:84 , srv1191323945:84 , srv1209279420:84 , srv1223713509:84 , srv1270228988:84 , srv1280986321:84 , srv1296168706:84 , srv130423643:84 , srv138853375:84 , srv1416352832:84 , srv145823731:84 , srv1502779866:84 , srv1527216169:84 , srv1559951217:84 , srv1606075359:84 , srv1607230784:84 , srv1643150158:84 , srv1654072595:84 , srv1657529043:84 , srv1659104754:84 , srv166630449:84 , srv1686853594:84 , srv1688583344:84 , srv1721314252:84 , srv1813990417:84 , srv1814534105:84 , srv1827821546:84 , srv1841480762:84 , srv1847004344:84 , srv1847509338:84 , srv1885644285:84 , srv1909213339:84 , srv196674924:84 , srv1994031907:84 , srv2001210251:84 , srv2027067909:84 , srv2040534305:84 , srv2053951669:84 , srv2067126166:84 , srv2125511565:84 , srv2138409723:84 , srv26109707:84 , srv270638664:84 , srv280237146:84 , srv281407866:84 , srv282274588:84 , srv318460076:84 , srv323650811:84 , srv356328359:84 , srv356796626:84 , srv397241486:84 , srv402138550:84 , srv404882606:84 , srv429291410:84 , srv436484516:84 , srv497295421:84 , srv502588795:84 , srv50315031:84 , srv530998855:84 , srv549886739:84 , srv564832497:84 , srv612701463:84 , srv675101007:84 , srv715081620:84 , srv745232943:84 , srv780759362:84 , srv806532160:84 , srv833787545:84 , srv836739667:84 , srv844628717:84 , srv86798776:84 , srv871291372:84 , srv874334184:84 , srv89622799:84 , srv966201220:84 , srv995194179:84 , srv1264097982:85 , srv1360731257:85 , srv1656858783:85 , srv1954371100:85 , srv2125338194:85 , srv335194478:85 , srv377436115:85 , srv654344976:85 , srv109969222:685 }
2023-11-11 06:55:14,670 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 100), ProcessCount=167 (was 167), AvailableMemoryMB=2455 (was 2935)
2023-11-11 06:55:14,677 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=108, ProcessCount=167, AvailableMemoryMB=2454
2023-11-11 06:55:14,678 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 06:55:14,678 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-11 06:55:14,693 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1083778597:84 , srv1257307411:84 , srv1273976000:84 , srv1320478422:84 , srv1331507216:84 , srv1465829517:84 , srv1592269920:84 , srv1684770870:84 , srv1812009896:84 , srv1828474838:84 , srv1828753477:84 , srv1936519470:84 , srv1974276188:84 , srv2110322132:84 , srv246501029:84 , srv314803976:84 , srv342809219:84 , srv378867183:84 , srv403263288:84 , srv413341697:84 , srv415655891:84 , srv422832040:84 , srv483577714:84 , srv571541028:84 , srv58106880:84 , srv718607275:84 , srv861602530:84 , srv862099079:84 , srv93077300:84 , srv1033571626:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-11 06:55:14,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:55:14,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1320478422=4, srv1936519470=12, srv1684770870=8, srv246501029=15, srv1592269920=7, srv862099079=28, srv1828474838=10, srv1033571626=0, srv378867183=18, srv1812009896=9, srv403263288=19, srv1828753477=11, srv422832040=22, srv861602530=27, srv58106880=25, srv2110322132=14, srv415655891=21, srv483577714=23, srv1083778597=1, srv93077300=29, srv413341697=20, srv314803976=16, srv342809219=17, srv718607275=26, srv1974276188=13, srv1257307411=2, srv571541028=24, srv1273976000=3, srv1331507216=5, srv1465829517=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:55:14,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:55:14,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:55:14,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:55:14,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:55:14,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:55:14,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:55:14,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:55:14,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:55:14,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:55:14,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-11 06:55:14,730 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1090171480481623, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:55:14,731 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912536552738043, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1090171480481623, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-11 06:57:14,715 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5324677 different iterations.  Found a solution that moves 1874 regions; Going from a computed imbalance of 0.9912536552738043 to a new imbalance of 4.374009487579354E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6940740740740741, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:57:14,720 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1033571626:90 , srv1083778597:90 , srv1257307411:90 , srv1273976000:90 , srv1320478422:90 , srv1331507216:90 , srv1465829517:90 , srv1592269920:90 , srv1684770870:90 , srv1812009896:90 , srv1828474838:90 , srv1828753477:90 , srv1936519470:90 , srv1974276188:90 , srv2110322132:90 , srv246501029:90 , srv314803976:90 , srv342809219:90 , srv378867183:90 , srv403263288:90 , srv413341697:90 , srv415655891:90 , srv422832040:90 , srv483577714:90 , srv571541028:90 , srv58106880:90 , srv718607275:90 , srv861602530:90 , srv862099079:90 , srv93077300:90 }
2023-11-11 06:57:14,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 06:57:14,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1320478422=4, srv1936519470=12, srv1684770870=8, srv246501029=15, srv1592269920=7, srv862099079=28, srv1828474838=10, srv1033571626=0, srv378867183=18, srv1812009896=9, srv403263288=19, srv1828753477=11, srv422832040=22, srv861602530=27, srv58106880=25, srv2110322132=14, srv415655891=21, srv483577714=23, srv1083778597=1, srv93077300=29, srv413341697=20, srv314803976=16, srv342809219=17, srv718607275=26, srv1974276188=13, srv1257307411=2, srv571541028=24, srv1273976000=3, srv1331507216=5, srv1465829517=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 06:57:14,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 06:57:14,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-11 06:57:14,738 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 06:57:14,738 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1033571626:90 , srv1083778597:90 , srv1257307411:90 , srv1273976000:90 , srv1320478422:90 , srv1331507216:90 , srv1465829517:90 , srv1592269920:90 , srv1684770870:90 , srv1812009896:90 , srv1828474838:90 , srv1828753477:90 , srv1936519470:90 , srv1974276188:90 , srv2110322132:90 , srv246501029:90 , srv314803976:90 , srv342809219:90 , srv378867183:90 , srv403263288:90 , srv413341697:90 , srv415655891:90 , srv422832040:90 , srv483577714:90 , srv571541028:90 , srv58106880:90 , srv718607275:90 , srv861602530:90 , srv862099079:90 , srv93077300:90 }
2023-11-11 06:57:14,750 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=153 (was 108) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=2617 (was 2454) - AvailableMemoryMB LEAK? -