2023-10-17 15:53:29,851 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-17 15:53:30,154 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-17 15:53:30,272 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-17 15:53:30,344 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 15:53:30,353 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-17 15:53:30,366 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=397, ProcessCount=169, AvailableMemoryMB=8155
2023-10-17 15:53:30,369 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 15:53:30,369 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-17 15:53:30,513 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017858090:84 , srv1027193833:84 , srv1040952784:84 , srv1056569615:84 , srv1068249085:84 , srv1121464770:84 , srv1129335989:84 , srv1165302692:84 , srv1168516888:84 , srv1172432448:84 , srv1192637070:84 , srv121766900:84 , srv1218630624:84 , srv1237181890:84 , srv1237668217:84 , srv1257460879:84 , srv1267282994:84 , srv1275259840:84 , srv1276422215:84 , srv1313854268:84 , srv1318758927:84 , srv1344016458:84 , srv1345367373:84 , srv1377795179:84 , srv1385255573:84 , srv1404562974:84 , srv1406273895:84 , srv1455022779:84 , srv1469963798:84 , srv1514113137:84 , srv1547638951:84 , srv1555194497:84 , srv156385379:84 , srv1564857457:84 , srv1575285742:84 , srv1585880511:84 , srv1602900618:84 , srv1603716403:84 , srv1604423824:84 , srv1613355076:84 , srv1630809669:84 , srv1660411510:84 , srv1665974155:84 , srv1666681962:84 , srv1675921795:84 , srv1710018493:84 , srv1730060853:84 , srv1748727052:84 , srv1770047140:84 , srv1772458651:84 , srv1775670315:84 , srv1780348858:84 , srv1820505557:84 , srv1856157143:84 , srv1856278751:84 , srv186404942:84 , srv1877698121:84 , srv1885939439:84 , srv1918001902:84 , srv1937169895:84 , srv1987731685:84 , srv199178561:84 , srv2081221604:84 , srv2085830837:84 , srv2085854148:84 , srv2087314035:84 , srv2144499283:84 , srv215433591:84 , srv239872967:84 , srv245940592:84 , srv247736714:84 , srv266729965:84 , srv305295212:84 , srv343037199:84 , srv38729600:84 , srv399389811:84 , srv437008448:84 , srv441119881:84 , srv443307206:84 , srv455091642:84 , srv481340023:84 , srv50819323:84 , srv518149520:84 , srv591124778:84 , srv616196053:84 , srv655970793:84 , srv663046524:84 , srv670771526:84 , srv760994038:84 , srv762839634:84 , srv776431613:84 , srv785419125:84 , srv874075074:84 , srv911077:84 , srv930600517:84 , srv934306604:84 , srv955945082:84 , srv973276502:84 , srv97724247:84 , srv1200445931:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-17 15:53:30,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:30,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv776431613=91, srv1267282994=17, srv1856157143=54, srv760994038=89, srv1027193833=1, srv1218630624=13, srv1237181890=14, srv955945082=97, srv247736714=71, srv1775670315=51, srv1675921795=45, srv1121464770=5, srv156385379=33, srv785419125=92, srv399389811=76, srv1345367373=23, srv1564857457=34, srv1602900618=37, srv1772458651=50, srv245940592=70, srv591124778=84, srv1604423824=39, srv186404942=56, srv305295212=73, srv1404562974=26, srv97724247=99, srv1200445931=11, srv38729600=75, srv911077=94, srv1585880511=36, srv1514113137=30, srv934306604=96, srv1068249085=4, srv1237668217=15, srv1257460879=16, srv663046524=87, srv1318758927=21, srv1937169895=60, srv1780348858=52, srv437008448=77, srv239872967=69, srv1275259840=18, srv199178561=62, srv1165302692=7, srv2087314035=66, srv1344016458=22, srv2085830837=64, srv1406273895=27, srv455091642=80, srv215433591=68, srv1555194497=32, srv1385255573=25, srv1748727052=48, srv2144499283=67, srv1455022779=28, srv1666681962=44, srv1730060853=47, srv50819323=82, srv1276422215=19, srv1469963798=29, srv1377795179=24, srv1040952784=2, srv443307206=79, srv1056569615=3, srv1313854268=20, srv1630809669=41, srv1603716403=38, srv655970793=86, srv973276502=98, srv1172432448=9, srv266729965=72, srv518149520=83, srv762839634=90, srv1918001902=59, srv1575285742=35, srv1665974155=43, srv1820505557=53, srv1660411510=42, srv930600517=95, srv481340023=81, srv441119881=78, srv1710018493=46, srv1168516888=8, srv1856278751=55, srv1129335989=6, srv343037199=74, srv874075074=93, srv1877698121=57, srv2081221604=63, srv1770047140=49, srv1017858090=0, srv1547638951=31, srv121766900=12, srv1192637070=10, srv670771526=88, srv1885939439=58, srv616196053=85, srv1613355076=40, srv1987731685=61, srv2085854148=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:30,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:30,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:30,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:30,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:30,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 15:53:30,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 15:53:30,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 15:53:30,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 15:53:30,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 15:53:30,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 15:53:30,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 15:53:30,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 15:53:30,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 15:53:30,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 15:53:30,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 15:53:30,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 15:53:30,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 15:53:30,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:30,864 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 15:53:30,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 15:53:31,688 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1002 ms to try 5000 different iterations.  Found a solution that moves 4693 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1782976649601951. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039817516156108984, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05796783039125546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5214444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039817516156108984, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17391666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:31,712 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv239872967:82 , srv1068249085:83 , srv1313854268:83 , srv1514113137:83 , srv1603716403:83 , srv1710018493:83 , srv1748727052:83 , srv1987731685:83 , srv343037199:83 , srv655970793:83 , srv1237668217:84 , srv1469963798:84 , srv1604423824:84 , srv1630809669:84 , srv1660411510:84 , srv1772458651:84 , srv1775670315:84 , srv1780348858:84 , srv2081221604:84 , srv2085854148:84 , srv245940592:84 , srv616196053:84 , srv1027193833:85 , srv1168516888:85 , srv1192637070:85 , srv1404562974:85 , srv1547638951:85 , srv156385379:85 , srv1665974155:85 , srv1666681962:85 , srv1675921795:85 , srv1856157143:85 , srv1877698121:85 , srv399389811:85 , srv591124778:85 , srv663046524:85 , srv776431613:85 , srv911077:85 , srv97724247:85 , srv1017858090:86 , srv1257460879:86 , srv1267282994:86 , srv1276422215:86 , srv1555194497:86 , srv1564857457:86 , srv1575285742:86 , srv1730060853:86 , srv199178561:86 , srv2085830837:86 , srv518149520:86 , srv785419125:86 , srv1040952784:87 , srv1056569615:87 , srv1121464770:87 , srv1129335989:87 , srv1218630624:87 , srv1237181890:87 , srv1275259840:87 , srv1377795179:87 , srv1455022779:87 , srv1770047140:87 , srv1937169895:87 , srv2087314035:87 , srv305295212:87 , srv437008448:87 , srv50819323:87 , srv930600517:87 , srv955945082:87 , srv973276502:87 , srv1172432448:88 , srv121766900:88 , srv1318758927:88 , srv1585880511:88 , srv1602900618:88 , srv1885939439:88 , srv2144499283:88 , srv38729600:88 , srv455091642:88 , srv481340023:88 , srv670771526:88 , srv760994038:88 , srv874075074:88 , srv1345367373:89 , srv1385255573:89 , srv1406273895:89 , srv1613355076:89 , srv215433591:89 , srv247736714:89 , srv266729965:89 , srv443307206:89 , srv762839634:89 , srv1165302692:90 , srv1344016458:90 , srv1856278751:90 , srv934306604:90 , srv1820505557:91 , srv441119881:91 , srv186404942:94 , srv1918001902:95 , srv1200445931:444 }
2023-10-17 15:53:31,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:31,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv776431613=91, srv1267282994=17, srv1856157143=54, srv760994038=89, srv1027193833=1, srv1218630624=13, srv1237181890=14, srv955945082=97, srv247736714=71, srv1775670315=51, srv1675921795=45, srv1121464770=5, srv156385379=33, srv785419125=92, srv399389811=76, srv1345367373=23, srv1564857457=34, srv1602900618=37, srv1772458651=50, srv245940592=70, srv591124778=84, srv1604423824=39, srv186404942=56, srv305295212=73, srv1404562974=26, srv97724247=99, srv1200445931=11, srv38729600=75, srv911077=94, srv1585880511=36, srv1514113137=30, srv934306604=96, srv1068249085=4, srv1237668217=15, srv1257460879=16, srv663046524=87, srv1318758927=21, srv1937169895=60, srv1780348858=52, srv437008448=77, srv239872967=69, srv1275259840=18, srv199178561=62, srv1165302692=7, srv2087314035=66, srv1344016458=22, srv2085830837=64, srv1406273895=27, srv455091642=80, srv215433591=68, srv1555194497=32, srv1385255573=25, srv1748727052=48, srv2144499283=67, srv1455022779=28, srv1666681962=44, srv1730060853=47, srv50819323=82, srv1276422215=19, srv1469963798=29, srv1377795179=24, srv1040952784=2, srv443307206=79, srv1056569615=3, srv1313854268=20, srv1630809669=41, srv1603716403=38, srv655970793=86, srv973276502=98, srv1172432448=9, srv266729965=72, srv518149520=83, srv762839634=90, srv1918001902=59, srv1575285742=35, srv1665974155=43, srv1820505557=53, srv1660411510=42, srv930600517=95, srv481340023=81, srv441119881=78, srv1710018493=46, srv1168516888=8, srv1856278751=55, srv1129335989=6, srv343037199=74, srv874075074=93, srv1877698121=57, srv2081221604=63, srv1770047140=49, srv1017858090=0, srv1547638951=31, srv121766900=12, srv1192637070=10, srv670771526=88, srv1885939439=58, srv616196053=85, srv1613355076=40, srv1987731685=61, srv2085854148=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:31,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:31,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:31,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:31,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:31,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:31,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 15:53:31,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 15:53:31,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 15:53:31,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 15:53:31,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 15:53:31,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 15:53:31,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 15:53:31,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 15:53:31,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 15:53:31,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039817516156108984, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05796783039125546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039817516156108984, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17391666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:31,881 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 15:53:31,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.178264803871841, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039817516156108984, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05796783039125546, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039817516156108984, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17391666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23591666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 15:53:32,135 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 365 ms to try 5000 different iterations.  Found a solution that moves 2602 regions; Going from a computed imbalance of 0.178264803871841 to a new imbalance of 0.014925119548673392. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021725652821549667); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040047670808041634, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2891111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021725652821549667); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:32,147 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1918001902:78 , srv1344016458:80 , srv1820505557:80 , srv186404942:80 , srv934306604:81 , srv1165302692:82 , srv1406273895:82 , srv266729965:82 , srv437008448:82 , srv441119881:82 , srv1040952784:83 , srv1172432448:83 , srv1237181890:83 , srv1345367373:83 , srv1385255573:83 , srv1555194497:83 , srv1585880511:83 , srv1613355076:83 , srv1856278751:83 , srv1877698121:83 , srv2087314035:83 , srv215433591:83 , srv481340023:83 , srv1017858090:84 , srv1129335989:84 , srv121766900:84 , srv1455022779:84 , srv1575285742:84 , srv2144499283:84 , srv247736714:84 , srv399389811:84 , srv670771526:84 , srv762839634:84 , srv973276502:84 , srv1056569615:85 , srv1218630624:85 , srv1257460879:85 , srv1267282994:85 , srv1276422215:85 , srv1377795179:85 , srv1564857457:85 , srv1602900618:85 , srv1730060853:85 , srv1856157143:85 , srv1885939439:85 , srv1937169895:85 , srv305295212:85 , srv38729600:85 , srv50819323:85 , srv591124778:85 , srv663046524:85 , srv1121464770:86 , srv1275259840:86 , srv1318758927:86 , srv199178561:86 , srv455091642:86 , srv760994038:86 , srv874075074:86 , srv955945082:86 , srv97724247:86 , srv1068249085:87 , srv1547638951:87 , srv1660411510:87 , srv1770047140:87 , srv1775670315:87 , srv239872967:87 , srv518149520:87 , srv785419125:87 , srv930600517:87 , srv1027193833:88 , srv1192637070:88 , srv1404562974:88 , srv1469963798:88 , srv1603716403:88 , srv1604423824:88 , srv1665974155:88 , srv1666681962:88 , srv1675921795:88 , srv1772458651:88 , srv2081221604:88 , srv2085854148:88 , srv245940592:88 , srv443307206:88 , srv776431613:88 , srv1168516888:89 , srv1237668217:89 , srv156385379:89 , srv1748727052:89 , srv1780348858:89 , srv2085830837:89 , srv616196053:89 , srv655970793:89 , srv911077:89 , srv1313854268:90 , srv1630809669:90 , srv1710018493:90 , srv1987731685:90 , srv343037199:90 , srv1514113137:92 , srv1200445931:523 }
2023-10-17 15:53:32,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:32,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv776431613=91, srv1267282994=17, srv1856157143=54, srv760994038=89, srv1027193833=1, srv1218630624=13, srv1237181890=14, srv955945082=97, srv247736714=71, srv1775670315=51, srv1675921795=45, srv1121464770=5, srv156385379=33, srv785419125=92, srv399389811=76, srv1345367373=23, srv1564857457=34, srv1602900618=37, srv1772458651=50, srv245940592=70, srv591124778=84, srv1604423824=39, srv186404942=56, srv305295212=73, srv1404562974=26, srv97724247=99, srv1200445931=11, srv38729600=75, srv911077=94, srv1585880511=36, srv1514113137=30, srv934306604=96, srv1068249085=4, srv1237668217=15, srv1257460879=16, srv663046524=87, srv1318758927=21, srv1937169895=60, srv1780348858=52, srv437008448=77, srv239872967=69, srv1275259840=18, srv199178561=62, srv1165302692=7, srv2087314035=66, srv1344016458=22, srv2085830837=64, srv1406273895=27, srv455091642=80, srv215433591=68, srv1555194497=32, srv1385255573=25, srv1748727052=48, srv2144499283=67, srv1455022779=28, srv1666681962=44, srv1730060853=47, srv50819323=82, srv1276422215=19, srv1469963798=29, srv1377795179=24, srv1040952784=2, srv443307206=79, srv1056569615=3, srv1313854268=20, srv1630809669=41, srv1603716403=38, srv655970793=86, srv973276502=98, srv1172432448=9, srv266729965=72, srv518149520=83, srv762839634=90, srv1918001902=59, srv1575285742=35, srv1665974155=43, srv1820505557=53, srv1660411510=42, srv930600517=95, srv481340023=81, srv441119881=78, srv1710018493=46, srv1168516888=8, srv1856278751=55, srv1129335989=6, srv343037199=74, srv874075074=93, srv1877698121=57, srv2081221604=63, srv1770047140=49, srv1017858090=0, srv1547638951=31, srv121766900=12, srv1192637070=10, srv670771526=88, srv1885939439=58, srv616196053=85, srv1613355076=40, srv1987731685=61, srv2085854148=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:32,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:32,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:32,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:32,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:32,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:32,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:32,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 15:53:32,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 15:53:32,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 15:53:32,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 15:53:32,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 15:53:32,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 15:53:32,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 15:53:32,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 15:53:32,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 15:53:32,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 15:53:32,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 15:53:32,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 15:53:32,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021725652821549667); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040047670808041634, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021725652821549667); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:32,321 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 15:53:32,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014906899955258214, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021725652821549667); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040047670808041634, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021725652821549667); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04325, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 15:53:32,546 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 336 ms to try 5000 different iterations.  Found a solution that moves 1010 regions; Going from a computed imbalance of 0.014906899955258214 to a new imbalance of 6.037897989912872E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004991575394184176); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008888888888888889); MoveCostFunction : (multiplier=7.0, imbalance=0.11222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004991575394184176); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:32,548 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv443307206:80 , srv1168516888:83 , srv1514113137:83 , srv186404942:83 , srv2085830837:83 , srv455091642:83 , srv911077:83 , srv1121464770:84 , srv1192637070:84 , srv1237668217:84 , srv1313854268:84 , srv1318758927:84 , srv156385379:84 , srv1630809669:84 , srv1665974155:84 , srv1748727052:84 , srv1780348858:84 , srv1918001902:84 , srv343037199:84 , srv38729600:84 , srv518149520:84 , srv616196053:84 , srv760994038:84 , srv776431613:84 , srv785419125:84 , srv874075074:84 , srv930600517:84 , srv1027193833:85 , srv1056569615:85 , srv1218630624:85 , srv1275259840:85 , srv1404562974:85 , srv1469963798:85 , srv1547638951:85 , srv1602900618:85 , srv1604423824:85 , srv1666681962:85 , srv1675921795:85 , srv1710018493:85 , srv1770047140:85 , srv1772458651:85 , srv1856278751:85 , srv1987731685:85 , srv2081221604:85 , srv2085854148:85 , srv215433591:85 , srv245940592:85 , srv247736714:85 , srv305295212:85 , srv441119881:85 , srv762839634:85 , srv955945082:85 , srv1017858090:86 , srv1129335989:86 , srv1172432448:86 , srv121766900:86 , srv1276422215:86 , srv1345367373:86 , srv1377795179:86 , srv1385255573:86 , srv1406273895:86 , srv1585880511:86 , srv1613355076:86 , srv1660411510:86 , srv1775670315:86 , srv1885939439:86 , srv1937169895:86 , srv199178561:86 , srv2144499283:86 , srv266729965:86 , srv50819323:86 , srv655970793:86 , srv670771526:86 , srv1068249085:87 , srv1165302692:87 , srv1257460879:87 , srv1267282994:87 , srv1455022779:87 , srv1555194497:87 , srv1564857457:87 , srv1730060853:87 , srv1820505557:87 , srv1856157143:87 , srv481340023:87 , srv663046524:87 , srv934306604:87 , srv973276502:87 , srv97724247:87 , srv1040952784:88 , srv1575285742:88 , srv1603716403:88 , srv2087314035:88 , srv239872967:88 , srv399389811:88 , srv437008448:88 , srv591124778:88 , srv1237181890:89 , srv1344016458:89 , srv1877698121:89 , srv1200445931:535 }
2023-10-17 15:53:32,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:32,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv776431613=91, srv1267282994=17, srv1856157143=54, srv760994038=89, srv1027193833=1, srv1218630624=13, srv1237181890=14, srv955945082=97, srv247736714=71, srv1775670315=51, srv1675921795=45, srv1121464770=5, srv156385379=33, srv785419125=92, srv399389811=76, srv1345367373=23, srv1564857457=34, srv1602900618=37, srv1772458651=50, srv245940592=70, srv591124778=84, srv1604423824=39, srv186404942=56, srv305295212=73, srv1404562974=26, srv97724247=99, srv1200445931=11, srv38729600=75, srv911077=94, srv1585880511=36, srv1514113137=30, srv934306604=96, srv1068249085=4, srv1237668217=15, srv1257460879=16, srv663046524=87, srv1318758927=21, srv1937169895=60, srv1780348858=52, srv437008448=77, srv239872967=69, srv1275259840=18, srv199178561=62, srv1165302692=7, srv2087314035=66, srv1344016458=22, srv2085830837=64, srv1406273895=27, srv455091642=80, srv215433591=68, srv1555194497=32, srv1385255573=25, srv1748727052=48, srv2144499283=67, srv1455022779=28, srv1666681962=44, srv1730060853=47, srv50819323=82, srv1276422215=19, srv1469963798=29, srv1377795179=24, srv1040952784=2, srv443307206=79, srv1056569615=3, srv1313854268=20, srv1630809669=41, srv1603716403=38, srv655970793=86, srv973276502=98, srv1172432448=9, srv266729965=72, srv518149520=83, srv762839634=90, srv1918001902=59, srv1575285742=35, srv1665974155=43, srv1820505557=53, srv1660411510=42, srv930600517=95, srv481340023=81, srv441119881=78, srv1710018493=46, srv1168516888=8, srv1856278751=55, srv1129335989=6, srv343037199=74, srv874075074=93, srv1877698121=57, srv2081221604=63, srv1770047140=49, srv1017858090=0, srv1547638951=31, srv121766900=12, srv1192637070=10, srv670771526=88, srv1885939439=58, srv616196053=85, srv1613355076=40, srv1987731685=61, srv2085854148=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:32,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 15:53:32,564 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 15:53:32,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 15:53:32,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 15:53:32,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 15:53:32,568 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 15:53:32,681 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004991575394184176); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008888888888888889); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004991575394184176); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:32,681 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-17 15:53:32,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.967176278347422E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004991575394184176); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008888888888888889); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004991575394184176); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-17 15:53:33,004 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 424 ms to try 5000 different iterations.  Found a solution that moves 269 regions; Going from a computed imbalance of 5.967176278347422E-4 to a new imbalance of 2.147697151189102E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.02988888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:33,005 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1165302692:83 , srv1237181890:83 , srv1344016458:83 , srv1603716403:83 , srv1730060853:83 , srv97724247:83 , srv1027193833:84 , srv1040952784:84 , srv1168516888:84 , srv121766900:84 , srv1267282994:84 , srv1275259840:84 , srv1318758927:84 , srv1345367373:84 , srv1377795179:84 , srv1385255573:84 , srv1404562974:84 , srv1455022779:84 , srv1514113137:84 , srv156385379:84 , srv1564857457:84 , srv1575285742:84 , srv1602900618:84 , srv1613355076:84 , srv1630809669:84 , srv1675921795:84 , srv1710018493:84 , srv1770047140:84 , srv1775670315:84 , srv1820505557:84 , srv1856278751:84 , srv1885939439:84 , srv1937169895:84 , srv1987731685:84 , srv199178561:84 , srv2085830837:84 , srv2087314035:84 , srv2144499283:84 , srv245940592:84 , srv247736714:84 , srv441119881:84 , srv481340023:84 , srv50819323:84 , srv591124778:84 , srv655970793:84 , srv670771526:84 , srv760994038:84 , srv762839634:84 , srv776431613:84 , srv874075074:84 , srv930600517:84 , srv934306604:84 , srv973276502:84 , srv1056569615:85 , srv1068249085:85 , srv1121464770:85 , srv1129335989:85 , srv1172432448:85 , srv1192637070:85 , srv1218630624:85 , srv1237668217:85 , srv1257460879:85 , srv1276422215:85 , srv1313854268:85 , srv1406273895:85 , srv1469963798:85 , srv1547638951:85 , srv1585880511:85 , srv1604423824:85 , srv1660411510:85 , srv1665974155:85 , srv1666681962:85 , srv1772458651:85 , srv1780348858:85 , srv1856157143:85 , srv186404942:85 , srv1877698121:85 , srv1918001902:85 , srv2081221604:85 , srv215433591:85 , srv239872967:85 , srv266729965:85 , srv305295212:85 , srv343037199:85 , srv38729600:85 , srv399389811:85 , srv437008448:85 , srv443307206:85 , srv455091642:85 , srv518149520:85 , srv616196053:85 , srv663046524:85 , srv785419125:85 , srv911077:85 , srv955945082:85 , srv1017858090:86 , srv1555194497:86 , srv1748727052:86 , srv2085854148:86 , srv1200445931:640 }
2023-10-17 15:53:33,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:33,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv776431613=91, srv1267282994=17, srv1856157143=54, srv760994038=89, srv1027193833=1, srv1218630624=13, srv1237181890=14, srv955945082=97, srv247736714=71, srv1775670315=51, srv1675921795=45, srv1121464770=5, srv156385379=33, srv785419125=92, srv399389811=76, srv1345367373=23, srv1564857457=34, srv1602900618=37, srv1772458651=50, srv245940592=70, srv591124778=84, srv1604423824=39, srv186404942=56, srv305295212=73, srv1404562974=26, srv97724247=99, srv1200445931=11, srv38729600=75, srv911077=94, srv1585880511=36, srv1514113137=30, srv934306604=96, srv1068249085=4, srv1237668217=15, srv1257460879=16, srv663046524=87, srv1318758927=21, srv1937169895=60, srv1780348858=52, srv437008448=77, srv239872967=69, srv1275259840=18, srv199178561=62, srv1165302692=7, srv2087314035=66, srv1344016458=22, srv2085830837=64, srv1406273895=27, srv455091642=80, srv215433591=68, srv1555194497=32, srv1385255573=25, srv1748727052=48, srv2144499283=67, srv1455022779=28, srv1666681962=44, srv1730060853=47, srv50819323=82, srv1276422215=19, srv1469963798=29, srv1377795179=24, srv1040952784=2, srv443307206=79, srv1056569615=3, srv1313854268=20, srv1630809669=41, srv1603716403=38, srv655970793=86, srv973276502=98, srv1172432448=9, srv266729965=72, srv518149520=83, srv762839634=90, srv1918001902=59, srv1575285742=35, srv1665974155=43, srv1820505557=53, srv1660411510=42, srv930600517=95, srv481340023=81, srv441119881=78, srv1710018493=46, srv1168516888=8, srv1856278751=55, srv1129335989=6, srv343037199=74, srv874075074=93, srv1877698121=57, srv2081221604=63, srv1770047140=49, srv1017858090=0, srv1547638951=31, srv121766900=12, srv1192637070=10, srv670771526=88, srv1885939439=58, srv616196053=85, srv1613355076=40, srv1987731685=61, srv2085854148=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:33,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-17 15:53:33,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-17 15:53:33,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-17 15:53:33,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-17 15:53:33,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-17 15:53:33,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-17 15:53:33,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-17 15:53:33,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-17 15:53:33,138 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.959339325336565E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:33,139 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1165302692:83 , srv1237181890:83 , srv1344016458:83 , srv1603716403:83 , srv1730060853:83 , srv97724247:83 , srv1027193833:84 , srv1040952784:84 , srv1168516888:84 , srv121766900:84 , srv1267282994:84 , srv1275259840:84 , srv1318758927:84 , srv1345367373:84 , srv1377795179:84 , srv1385255573:84 , srv1404562974:84 , srv1455022779:84 , srv1514113137:84 , srv156385379:84 , srv1564857457:84 , srv1575285742:84 , srv1602900618:84 , srv1613355076:84 , srv1630809669:84 , srv1675921795:84 , srv1710018493:84 , srv1770047140:84 , srv1775670315:84 , srv1820505557:84 , srv1856278751:84 , srv1885939439:84 , srv1937169895:84 , srv1987731685:84 , srv199178561:84 , srv2085830837:84 , srv2087314035:84 , srv2144499283:84 , srv245940592:84 , srv247736714:84 , srv441119881:84 , srv481340023:84 , srv50819323:84 , srv591124778:84 , srv655970793:84 , srv670771526:84 , srv760994038:84 , srv762839634:84 , srv776431613:84 , srv874075074:84 , srv930600517:84 , srv934306604:84 , srv973276502:84 , srv1056569615:85 , srv1068249085:85 , srv1121464770:85 , srv1129335989:85 , srv1172432448:85 , srv1192637070:85 , srv1218630624:85 , srv1237668217:85 , srv1257460879:85 , srv1276422215:85 , srv1313854268:85 , srv1406273895:85 , srv1469963798:85 , srv1547638951:85 , srv1585880511:85 , srv1604423824:85 , srv1660411510:85 , srv1665974155:85 , srv1666681962:85 , srv1772458651:85 , srv1780348858:85 , srv1856157143:85 , srv186404942:85 , srv1877698121:85 , srv1918001902:85 , srv2081221604:85 , srv215433591:85 , srv239872967:85 , srv266729965:85 , srv305295212:85 , srv343037199:85 , srv38729600:85 , srv399389811:85 , srv437008448:85 , srv443307206:85 , srv455091642:85 , srv518149520:85 , srv616196053:85 , srv663046524:85 , srv785419125:85 , srv911077:85 , srv955945082:85 , srv1017858090:86 , srv1555194497:86 , srv1748727052:86 , srv2085854148:86 , srv1200445931:640 }
2023-10-17 15:53:33,172 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=397 (was 397), ProcessCount=169 (was 169), AvailableMemoryMB=7661 (was 8155)
2023-10-17 15:53:33,180 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=397, ProcessCount=169, AvailableMemoryMB=7661
2023-10-17 15:53:33,181 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-17 15:53:33,181 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-17 15:53:33,193 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv106942588:84 , srv1347185323:84 , srv1416374977:84 , srv1539216280:84 , srv1629799640:84 , srv1762722745:84 , srv1781986574:84 , srv180276171:84 , srv1846734421:84 , srv1898129060:84 , srv1918282090:84 , srv1932016695:84 , srv217094604:84 , srv23282482:84 , srv238700744:84 , srv386435289:84 , srv39307590:84 , srv448319893:84 , srv470388413:84 , srv519346303:84 , srv529067052:84 , srv540308234:84 , srv548122133:84 , srv570116757:84 , srv572942207:84 , srv682491345:84 , srv769544046:84 , srv795857417:84 , srv963361484:84 , srv1343247075:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-17 15:53:33,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:53:33,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv217094604=13, srv529067052=21, srv1846734421=9, srv1918282090=11, srv540308234=22, srv238700744=15, srv1898129060=10, srv39307590=17, srv963361484=29, srv180276171=8, srv548122133=23, srv769544046=27, srv470388413=19, srv682491345=26, srv386435289=16, srv572942207=25, srv519346303=20, srv570116757=24, srv106942588=0, srv1343247075=1, srv1629799640=5, srv795857417=28, srv23282482=14, srv1539216280=4, srv448319893=18, srv1932016695=12, srv1416374977=3, srv1347185323=2, srv1762722745=6, srv1781986574=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:53:33,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:53:33,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:53:33,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-17 15:53:33,236 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1769683805642546, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:53:33,236 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912750664852886, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1769683805642546, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-17 15:55:33,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6481368 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912750664852886 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:55:33,221 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv106942588:90 , srv1343247075:90 , srv1347185323:90 , srv1416374977:90 , srv1539216280:90 , srv1629799640:90 , srv1762722745:90 , srv1781986574:90 , srv180276171:90 , srv1846734421:90 , srv1898129060:90 , srv1918282090:90 , srv1932016695:90 , srv217094604:90 , srv23282482:90 , srv238700744:90 , srv386435289:90 , srv39307590:90 , srv448319893:90 , srv470388413:90 , srv519346303:90 , srv529067052:90 , srv540308234:90 , srv548122133:90 , srv570116757:90 , srv572942207:90 , srv682491345:90 , srv769544046:90 , srv795857417:90 , srv963361484:90 }
2023-10-17 15:55:33,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-17 15:55:33,222 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv217094604=13, srv529067052=21, srv1846734421=9, srv1918282090=11, srv540308234=22, srv238700744=15, srv1898129060=10, srv39307590=17, srv963361484=29, srv180276171=8, srv548122133=23, srv769544046=27, srv470388413=19, srv682491345=26, srv386435289=16, srv572942207=25, srv519346303=20, srv570116757=24, srv106942588=0, srv1343247075=1, srv1629799640=5, srv795857417=28, srv23282482=14, srv1539216280=4, srv448319893=18, srv1932016695=12, srv1416374977=3, srv1347185323=2, srv1762722745=6, srv1781986574=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-17 15:55:33,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-17 15:55:33,226 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-17 15:55:33,227 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-17 15:55:33,241 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-17 15:55:33,241 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv106942588:90 , srv1343247075:90 , srv1347185323:90 , srv1416374977:90 , srv1539216280:90 , srv1629799640:90 , srv1762722745:90 , srv1781986574:90 , srv180276171:90 , srv1846734421:90 , srv1898129060:90 , srv1918282090:90 , srv1932016695:90 , srv217094604:90 , srv23282482:90 , srv238700744:90 , srv386435289:90 , srv39307590:90 , srv448319893:90 , srv470388413:90 , srv519346303:90 , srv529067052:90 , srv540308234:90 , srv548122133:90 , srv570116757:90 , srv572942207:90 , srv682491345:90 , srv769544046:90 , srv795857417:90 , srv963361484:90 }
2023-10-17 15:55:33,256 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 397), ProcessCount=170 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=7584 (was 7661)