2024-03-25 14:55:06,488 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-03-25 14:55:06,739 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-03-25 14:55:06,831 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-03-25 14:55:06,883 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-25 14:55:06,889 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-25 14:55:06,902 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=176, AvailableMemoryMB=6878
2024-03-25 14:55:06,904 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-25 14:55:06,904 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-25 14:55:07,028 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015132877:84 , srv1023856486:84 , srv1024446425:84 , srv1039728435:84 , srv1040352602:84 , srv1043023780:84 , srv1044728531:84 , srv1070369341:84 , srv1075687387:84 , srv1082149544:84 , srv1108434608:84 , srv1132271860:84 , srv1138499843:84 , srv1147800320:84 , srv1161723955:84 , srv1165299539:84 , srv1265207529:84 , srv1276608768:84 , srv1318499852:84 , srv1320467955:84 , srv133933247:84 , srv1353025001:84 , srv1358635478:84 , srv1370860144:84 , srv1371683708:84 , srv1385444996:84 , srv1390324517:84 , srv1441052482:84 , srv1535479089:84 , srv154672469:84 , srv1565073996:84 , srv1574539351:84 , srv1578663575:84 , srv159964564:84 , srv161566515:84 , srv1619693980:84 , srv1622945427:84 , srv1623866860:84 , srv1681728487:84 , srv1699248035:84 , srv1706273767:84 , srv1720759885:84 , srv1740389786:84 , srv1765515532:84 , srv1803976262:84 , srv180966965:84 , srv1815209476:84 , srv1858378783:84 , srv1864702998:84 , srv1917419578:84 , srv1934542764:84 , srv1941521871:84 , srv1948691152:84 , srv1960107501:84 , srv1974272584:84 , srv1986100978:84 , srv1992104268:84 , srv2041155714:84 , srv204578660:84 , srv2128014377:84 , srv2146238614:84 , srv219923894:84 , srv241759868:84 , srv317025557:84 , srv319392322:84 , srv346913158:84 , srv350130366:84 , srv357049221:84 , srv364111656:84 , srv424839205:84 , srv435032217:84 , srv461305475:84 , srv484540859:84 , srv497224587:84 , srv521573494:84 , srv533033296:84 , srv541612512:84 , srv562380573:84 , srv590648757:84 , srv60026017:84 , srv635847192:84 , srv639853310:84 , srv640620980:84 , srv674149665:84 , srv703279924:84 , srv73949780:84 , srv745213781:84 , srv768765556:84 , srv769380602:84 , srv795159805:84 , srv813952778:84 , srv821709277:84 , srv821770725:84 , srv87962973:84 , srv895746629:84 , srv899938462:84 , srv932607244:84 , srv933829728:84 , srv999365400:84 , srv427768657:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-03-25 14:55:07,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:07,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:07,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:07,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:07,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:07,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:07,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:07,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:07,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:07,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:07,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:07,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:07,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:07,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:07,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:07,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:07,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:07,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:07,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:07,413 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:07,414 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-25 14:55:07,414 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-25 14:55:07,761 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 614 ms to try 5000 different iterations.  Found a solution that moves 4636 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1871721031465133. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04214092237742669, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05706239324444243, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5151111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04214092237742669, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18241666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24941666666666668, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:07,772 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1082149544:78 , srv640620980:79 , srv1024446425:80 , srv1623866860:80 , srv769380602:80 , srv1441052482:81 , srv1720759885:81 , srv1043023780:82 , srv541612512:82 , srv932607244:82 , srv1353025001:83 , srv1371683708:83 , srv1578663575:83 , srv1986100978:83 , srv364111656:83 , srv745213781:83 , srv795159805:83 , srv1147800320:84 , srv1276608768:84 , srv1320467955:84 , srv1370860144:84 , srv1681728487:84 , srv1699248035:84 , srv180966965:84 , srv2041155714:84 , srv2128014377:84 , srv317025557:84 , srv357049221:84 , srv461305475:84 , srv895746629:84 , srv1039728435:85 , srv1706273767:85 , srv1948691152:85 , srv1960107501:85 , srv319392322:85 , srv346913158:85 , srv484540859:85 , srv1044728531:86 , srv1165299539:86 , srv1385444996:86 , srv1535479089:86 , srv1815209476:86 , srv219923894:86 , srv350130366:86 , srv424839205:86 , srv497224587:86 , srv533033296:86 , srv562380573:86 , srv635847192:86 , srv73949780:86 , srv899938462:86 , srv1108434608:87 , srv1138499843:87 , srv1265207529:87 , srv1358635478:87 , srv1619693980:87 , srv1622945427:87 , srv1765515532:87 , srv1974272584:87 , srv204578660:87 , srv521573494:87 , srv590648757:87 , srv60026017:87 , srv639853310:87 , srv813952778:87 , srv821709277:87 , srv821770725:87 , srv933829728:87 , srv1075687387:88 , srv1132271860:88 , srv133933247:88 , srv154672469:88 , srv159964564:88 , srv1740389786:88 , srv1803976262:88 , srv1941521871:88 , srv1992104268:88 , srv435032217:88 , srv674149665:88 , srv1070369341:89 , srv1318499852:89 , srv1565073996:89 , srv1574539351:89 , srv1858378783:89 , srv1917419578:89 , srv1934542764:89 , srv2146238614:89 , srv999365400:89 , srv1023856486:90 , srv1161723955:90 , srv1390324517:90 , srv1864702998:90 , srv241759868:90 , srv703279924:90 , srv768765556:90 , srv87962973:91 , srv161566515:93 , srv1015132877:94 , srv1040352602:103 , srv427768657:464 }
2024-03-25 14:55:07,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:07,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:07,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:07,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:07,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:07,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:07,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:07,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:07,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:07,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:07,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:07,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:07,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:07,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:07,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:07,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:07,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:07,953 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04214092237742669, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05706239324444243, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04214092237742669, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18241666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24941666666666668, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:07,953 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-25 14:55:07,953 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1871396411806898, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04214092237742669, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05706239324444243, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04214092237742669, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18241666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24941666666666668, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-25 14:55:08,142 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 303 ms to try 5000 different iterations.  Found a solution that moves 2669 regions; Going from a computed imbalance of 0.1871396411806898 to a new imbalance of 0.015063466476423449. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024519322110382907); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037688545834154354, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29655555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024519322110382907); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,150 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1040352602:69 , srv1015132877:79 , srv161566515:79 , srv1858378783:80 , srv1864702998:80 , srv768765556:80 , srv1390324517:81 , srv1565073996:81 , srv241759868:81 , srv703279924:81 , srv1023856486:82 , srv1070369341:82 , srv1132271860:82 , srv1161723955:82 , srv1574539351:82 , srv1803976262:82 , srv1934542764:82 , srv674149665:82 , srv87962973:82 , srv999365400:82 , srv159964564:83 , srv1619693980:83 , srv1917419578:83 , srv1075687387:84 , srv1622945427:84 , srv1992104268:84 , srv435032217:84 , srv497224587:84 , srv521573494:84 , srv60026017:84 , srv1044728531:85 , srv1165299539:85 , srv133933247:85 , srv1358635478:85 , srv1740389786:85 , srv1765515532:85 , srv1815209476:85 , srv204578660:85 , srv2146238614:85 , srv219923894:85 , srv533033296:85 , srv590648757:85 , srv821709277:85 , srv821770725:85 , srv899938462:85 , srv1138499843:86 , srv1147800320:86 , srv1370860144:86 , srv1385444996:86 , srv154672469:86 , srv1941521871:86 , srv1948691152:86 , srv1960107501:86 , srv350130366:86 , srv424839205:86 , srv635847192:86 , srv639853310:86 , srv933829728:86 , srv1039728435:87 , srv1108434608:87 , srv1265207529:87 , srv1276608768:87 , srv1320467955:87 , srv1371683708:87 , srv1535479089:87 , srv1681728487:87 , srv180966965:87 , srv484540859:87 , srv541612512:87 , srv562380573:87 , srv1318499852:88 , srv1353025001:88 , srv1578663575:88 , srv1699248035:88 , srv1706273767:88 , srv2041155714:88 , srv319392322:88 , srv357049221:88 , srv461305475:88 , srv73949780:88 , srv813952778:88 , srv895746629:88 , srv932607244:88 , srv1441052482:89 , srv1986100978:89 , srv2128014377:89 , srv346913158:89 , srv745213781:89 , srv795159805:89 , srv1024446425:90 , srv1043023780:90 , srv1974272584:90 , srv317025557:90 , srv364111656:90 , srv769380602:90 , srv1623866860:92 , srv1720759885:92 , srv640620980:92 , srv1082149544:93 , srv427768657:528 }
2024-03-25 14:55:08,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:08,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:08,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:08,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:08,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:08,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:08,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:08,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:08,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:08,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:08,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:08,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:08,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:08,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:08,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024519322110382907); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037688545834154354, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024519322110382907); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,259 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-25 14:55:08,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015044777738981054, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.024519322110382907); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037688545834154354, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.024519322110382907); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011833333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-25 14:55:08,414 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 228 ms to try 5000 different iterations.  Found a solution that moves 982 regions; Going from a computed imbalance of 0.015044777738981054 to a new imbalance of 9.065448234713305E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007901059201196417); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.10911111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007901059201196417); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,415 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1318499852:80 , srv1974272584:80 , srv317025557:83 , srv346913158:83 , srv813952778:83 , srv1015132877:84 , srv1040352602:84 , srv1108434608:84 , srv1265207529:84 , srv1535479089:84 , srv154672469:84 , srv1706273767:84 , srv1720759885:84 , srv1941521871:84 , srv1986100978:84 , srv2128014377:84 , srv2146238614:84 , srv319392322:84 , srv364111656:84 , srv639853310:84 , srv73949780:84 , srv87962973:84 , srv1023856486:85 , srv1039728435:85 , srv1043023780:85 , srv1138499843:85 , srv133933247:85 , srv1358635478:85 , srv1390324517:85 , srv161566515:85 , srv1623866860:85 , srv1681728487:85 , srv1699248035:85 , srv1740389786:85 , srv1765515532:85 , srv1815209476:85 , srv1917419578:85 , srv1960107501:85 , srv1992104268:85 , srv2041155714:85 , srv204578660:85 , srv241759868:85 , srv350130366:85 , srv357049221:85 , srv424839205:85 , srv435032217:85 , srv562380573:85 , srv590648757:85 , srv60026017:85 , srv745213781:85 , srv795159805:85 , srv821709277:85 , srv895746629:85 , srv933829728:85 , srv1024446425:86 , srv1044728531:86 , srv1070369341:86 , srv1075687387:86 , srv1165299539:86 , srv1276608768:86 , srv1320467955:86 , srv1385444996:86 , srv1578663575:86 , srv159964564:86 , srv219923894:86 , srv461305475:86 , srv484540859:86 , srv521573494:86 , srv533033296:86 , srv635847192:86 , srv640620980:86 , srv703279924:86 , srv821770725:86 , srv899938462:86 , srv999365400:86 , srv1082149544:87 , srv1132271860:87 , srv1147800320:87 , srv1161723955:87 , srv1619693980:87 , srv1622945427:87 , srv180966965:87 , srv1858378783:87 , srv1934542764:87 , srv1948691152:87 , srv497224587:87 , srv768765556:87 , srv769380602:87 , srv932607244:87 , srv1370860144:88 , srv1371683708:88 , srv1565073996:88 , srv1574539351:88 , srv1803976262:88 , srv1864702998:88 , srv541612512:88 , srv674149665:88 , srv1353025001:89 , srv1441052482:90 , srv427768657:536 }
2024-03-25 14:55:08,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:08,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:08,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:08,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:08,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:08,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:08,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:08,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:08,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:08,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:08,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007901059201196417); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007901059201196417); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,516 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-25 14:55:08,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.99668712505264E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007901059201196417); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007901059201196417); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-25 14:55:08,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 292 ms to try 5000 different iterations.  Found a solution that moves 282 regions; Going from a computed imbalance of 8.99668712505264E-4 to a new imbalance of 4.252695576316041E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03133333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,737 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1161723955:82 , srv1353025001:82 , srv1441052482:83 , srv1574539351:83 , srv1075687387:84 , srv1082149544:84 , srv1108434608:84 , srv1138499843:84 , srv1265207529:84 , srv1320467955:84 , srv133933247:84 , srv1370860144:84 , srv1385444996:84 , srv154672469:84 , srv1565073996:84 , srv1578663575:84 , srv1619693980:84 , srv1622945427:84 , srv1740389786:84 , srv1803976262:84 , srv180966965:84 , srv1864702998:84 , srv1934542764:84 , srv1941521871:84 , srv1948691152:84 , srv2146238614:84 , srv317025557:84 , srv461305475:84 , srv484540859:84 , srv635847192:84 , srv73949780:84 , srv821770725:84 , srv933829728:84 , srv1015132877:85 , srv1023856486:85 , srv1039728435:85 , srv1043023780:85 , srv1044728531:85 , srv1070369341:85 , srv1132271860:85 , srv1147800320:85 , srv1165299539:85 , srv1276608768:85 , srv1318499852:85 , srv1371683708:85 , srv1535479089:85 , srv159964564:85 , srv161566515:85 , srv1623866860:85 , srv1699248035:85 , srv1706273767:85 , srv1720759885:85 , srv1765515532:85 , srv1917419578:85 , srv1960107501:85 , srv1974272584:85 , srv1992104268:85 , srv2041155714:85 , srv204578660:85 , srv2128014377:85 , srv219923894:85 , srv319392322:85 , srv350130366:85 , srv357049221:85 , srv364111656:85 , srv424839205:85 , srv435032217:85 , srv497224587:85 , srv521573494:85 , srv533033296:85 , srv541612512:85 , srv562380573:85 , srv590648757:85 , srv60026017:85 , srv639853310:85 , srv640620980:85 , srv674149665:85 , srv703279924:85 , srv745213781:85 , srv768765556:85 , srv769380602:85 , srv795159805:85 , srv813952778:85 , srv821709277:85 , srv87962973:85 , srv895746629:85 , srv899938462:85 , srv932607244:85 , srv999365400:85 , srv1024446425:86 , srv1040352602:86 , srv1358635478:86 , srv1390324517:86 , srv1681728487:86 , srv1815209476:86 , srv1858378783:86 , srv1986100978:86 , srv241759868:86 , srv346913158:86 , srv427768657:614 }
2024-03-25 14:55:08,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:08,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:08,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:08,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:08,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:08,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:08,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:08,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:08,837 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,837 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-25 14:55:08,837 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.055234955905575E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-25 14:55:08,984 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 220 ms to try 5000 different iterations.  Found a solution that moves 31 regions; Going from a computed imbalance of 4.055234955905575E-5 to a new imbalance of 6.249145379186682E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0034444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:08,985 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1358635478:83 , srv1371683708:83 , srv1441052482:83 , srv346913158:83 , srv562380573:83 , srv674149665:83 , srv813952778:83 , srv1015132877:84 , srv1023856486:84 , srv1024446425:84 , srv1039728435:84 , srv1040352602:84 , srv1043023780:84 , srv1044728531:84 , srv1070369341:84 , srv1075687387:84 , srv1082149544:84 , srv1108434608:84 , srv1132271860:84 , srv1138499843:84 , srv1147800320:84 , srv1165299539:84 , srv1265207529:84 , srv1276608768:84 , srv1318499852:84 , srv133933247:84 , srv1353025001:84 , srv1370860144:84 , srv1385444996:84 , srv1390324517:84 , srv1535479089:84 , srv154672469:84 , srv1565073996:84 , srv1574539351:84 , srv159964564:84 , srv161566515:84 , srv1622945427:84 , srv1623866860:84 , srv1681728487:84 , srv1699248035:84 , srv1706273767:84 , srv1720759885:84 , srv1740389786:84 , srv1765515532:84 , srv1803976262:84 , srv180966965:84 , srv1815209476:84 , srv1858378783:84 , srv1864702998:84 , srv1917419578:84 , srv1934542764:84 , srv1941521871:84 , srv1948691152:84 , srv1974272584:84 , srv1986100978:84 , srv1992104268:84 , srv2041155714:84 , srv204578660:84 , srv2128014377:84 , srv2146238614:84 , srv219923894:84 , srv241759868:84 , srv319392322:84 , srv350130366:84 , srv357049221:84 , srv364111656:84 , srv424839205:84 , srv435032217:84 , srv461305475:84 , srv484540859:84 , srv497224587:84 , srv521573494:84 , srv533033296:84 , srv541612512:84 , srv590648757:84 , srv635847192:84 , srv639853310:84 , srv640620980:84 , srv703279924:84 , srv73949780:84 , srv745213781:84 , srv768765556:84 , srv769380602:84 , srv795159805:84 , srv821709277:84 , srv821770725:84 , srv87962973:84 , srv895746629:84 , srv899938462:84 , srv932607244:84 , srv933829728:84 , srv999365400:84 , srv1161723955:85 , srv1320467955:85 , srv1578663575:85 , srv1619693980:85 , srv1960107501:85 , srv317025557:85 , srv60026017:85 , srv427768657:684 }
2024-03-25 14:55:08,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:08,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv154672469=29, srv1681728487=38, srv1353025001=21, srv1276608768=17, srv497224587=74, srv1740389786=42, srv899938462=96, srv1147800320=13, srv1082149544=9, srv1023856486=1, srv1960107501=53, srv180966965=45, srv484540859=73, srv821709277=92, srv562380573=78, srv1992104268=56, srv1371683708=24, srv1044728531=6, srv1390324517=26, srv1619693980=35, srv769380602=89, srv1917419578=49, srv999365400=99, srv1441052482=27, srv219923894=61, srv745213781=87, srv357049221=67, srv1040352602=4, srv1934542764=50, srv1043023780=5, srv435032217=71, srv1803976262=44, srv319392322=64, srv1574539351=31, srv1622945427=36, srv1165299539=15, srv241759868=62, srv1138499843=12, srv2041155714=57, srv1535479089=28, srv795159805=90, srv813952778=91, srv73949780=86, srv1578663575=32, srv161566515=34, srv317025557=63, srv1161723955=14, srv1720759885=41, srv590648757=79, srv933829728=98, srv1070369341=7, srv1974272584=54, srv2128014377=59, srv2146238614=60, srv461305475=72, srv635847192=81, srv1370860144=23, srv1623866860=37, srv533033296=76, srv1132271860=11, srv204578660=58, srv1706273767=40, srv159964564=33, srv1699248035=39, srv1320467955=19, srv639853310=82, srv1358635478=22, srv424839205=69, srv1565073996=30, srv1108434608=10, srv1024446425=2, srv1265207529=16, srv895746629=95, srv703279924=85, srv1941521871=51, srv521573494=75, srv87962973=94, srv1318499852=18, srv1815209476=46, srv1948691152=52, srv1385444996=25, srv133933247=20, srv346913158=65, srv364111656=68, srv640620980=83, srv1015132877=0, srv541612512=77, srv1858378783=47, srv674149665=84, srv1864702998=48, srv427768657=70, srv932607244=97, srv1039728435=3, srv350130366=66, srv821770725=93, srv1075687387=8, srv768765556=88, srv60026017=80, srv1765515532=43, srv1986100978=55} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:08,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-25 14:55:08,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-25 14:55:08,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-25 14:55:09,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-25 14:55:09,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-25 14:55:09,079 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.032078739728369E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0013400504203456161); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:09,079 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1358635478:83 , srv1371683708:83 , srv1441052482:83 , srv346913158:83 , srv562380573:83 , srv674149665:83 , srv813952778:83 , srv1015132877:84 , srv1023856486:84 , srv1024446425:84 , srv1039728435:84 , srv1040352602:84 , srv1043023780:84 , srv1044728531:84 , srv1070369341:84 , srv1075687387:84 , srv1082149544:84 , srv1108434608:84 , srv1132271860:84 , srv1138499843:84 , srv1147800320:84 , srv1165299539:84 , srv1265207529:84 , srv1276608768:84 , srv1318499852:84 , srv133933247:84 , srv1353025001:84 , srv1370860144:84 , srv1385444996:84 , srv1390324517:84 , srv1535479089:84 , srv154672469:84 , srv1565073996:84 , srv1574539351:84 , srv159964564:84 , srv161566515:84 , srv1622945427:84 , srv1623866860:84 , srv1681728487:84 , srv1699248035:84 , srv1706273767:84 , srv1720759885:84 , srv1740389786:84 , srv1765515532:84 , srv1803976262:84 , srv180966965:84 , srv1815209476:84 , srv1858378783:84 , srv1864702998:84 , srv1917419578:84 , srv1934542764:84 , srv1941521871:84 , srv1948691152:84 , srv1974272584:84 , srv1986100978:84 , srv1992104268:84 , srv2041155714:84 , srv204578660:84 , srv2128014377:84 , srv2146238614:84 , srv219923894:84 , srv241759868:84 , srv319392322:84 , srv350130366:84 , srv357049221:84 , srv364111656:84 , srv424839205:84 , srv435032217:84 , srv461305475:84 , srv484540859:84 , srv497224587:84 , srv521573494:84 , srv533033296:84 , srv541612512:84 , srv590648757:84 , srv635847192:84 , srv639853310:84 , srv640620980:84 , srv703279924:84 , srv73949780:84 , srv745213781:84 , srv768765556:84 , srv769380602:84 , srv795159805:84 , srv821709277:84 , srv821770725:84 , srv87962973:84 , srv895746629:84 , srv899938462:84 , srv932607244:84 , srv933829728:84 , srv999365400:84 , srv1161723955:85 , srv1320467955:85 , srv1578663575:85 , srv1619693980:85 , srv1960107501:85 , srv317025557:85 , srv60026017:85 , srv427768657:684 }
2024-03-25 14:55:09,101 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 173), ProcessCount=176 (was 176), AvailableMemoryMB=6389 (was 6878)
2024-03-25 14:55:09,109 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=176, AvailableMemoryMB=6389
2024-03-25 14:55:09,110 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-25 14:55:09,110 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-03-25 14:55:09,117 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014245461:84 , srv1059982332:84 , srv1062750081:84 , srv1147796118:84 , srv1262739713:84 , srv1288412640:84 , srv1375109529:84 , srv1390758213:84 , srv159328646:84 , srv1637166630:84 , srv172383669:84 , srv1781666645:84 , srv1839734429:84 , srv1919913841:84 , srv1923607650:84 , srv1972239327:84 , srv222548240:84 , srv287882790:84 , srv30201123:84 , srv394209133:84 , srv433260866:84 , srv632979368:84 , srv659904540:84 , srv718418587:84 , srv776290519:84 , srv790770343:84 , srv819014074:84 , srv865018006:84 , srv884354:84 , srv567095537:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-03-25 14:55:09,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:55:09,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1375109529=6, srv776290519=25, srv159328646=8, srv222548240=16, srv659904540=23, srv433260866=20, srv865018006=28, srv1147796118=3, srv1919913841=13, srv1972239327=15, srv632979368=22, srv1637166630=9, srv790770343=26, srv30201123=18, srv172383669=10, srv819014074=27, srv1288412640=5, srv287882790=17, srv567095537=21, srv1390758213=7, srv1059982332=1, srv1839734429=12, srv1062750081=2, srv718418587=24, srv394209133=19, srv884354=29, srv1781666645=11, srv1262739713=4, srv1923607650=14, srv1014245461=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:55:09,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:55:09,126 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-25 14:55:09,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0946247121478567, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:55:09,141 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912491202642477, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0946247121478567, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-03-25 14:57:09,129 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9116641 different iterations.  Found a solution that moves 1843 regions; Going from a computed imbalance of 0.9912491202642477 to a new imbalance of 4.3016539410932495E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6825925925925926, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:57:09,134 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1014245461:90 , srv1059982332:90 , srv1062750081:90 , srv1147796118:90 , srv1262739713:90 , srv1288412640:90 , srv1375109529:90 , srv1390758213:90 , srv159328646:90 , srv1637166630:90 , srv172383669:90 , srv1781666645:90 , srv1839734429:90 , srv1919913841:90 , srv1923607650:90 , srv1972239327:90 , srv222548240:90 , srv287882790:90 , srv30201123:90 , srv394209133:90 , srv433260866:90 , srv567095537:90 , srv632979368:90 , srv659904540:90 , srv718418587:90 , srv776290519:90 , srv790770343:90 , srv819014074:90 , srv865018006:90 , srv884354:90 }
2024-03-25 14:57:09,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-25 14:57:09,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1375109529=6, srv776290519=25, srv159328646=8, srv222548240=16, srv659904540=23, srv433260866=20, srv865018006=28, srv1147796118=3, srv1919913841=13, srv1972239327=15, srv632979368=22, srv1637166630=9, srv790770343=26, srv30201123=18, srv172383669=10, srv819014074=27, srv1288412640=5, srv287882790=17, srv567095537=21, srv1390758213=7, srv1059982332=1, srv1839734429=12, srv1062750081=2, srv718418587=24, srv394209133=19, srv884354=29, srv1781666645=11, srv1262739713=4, srv1923607650=14, srv1014245461=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-25 14:57:09,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-25 14:57:09,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-25 14:57:09,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-25 14:57:09,148 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-25 14:57:09,148 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1014245461:90 , srv1059982332:90 , srv1062750081:90 , srv1147796118:90 , srv1262739713:90 , srv1288412640:90 , srv1375109529:90 , srv1390758213:90 , srv159328646:90 , srv1637166630:90 , srv172383669:90 , srv1781666645:90 , srv1839734429:90 , srv1919913841:90 , srv1923607650:90 , srv1972239327:90 , srv222548240:90 , srv287882790:90 , srv30201123:90 , srv394209133:90 , srv433260866:90 , srv567095537:90 , srv632979368:90 , srv659904540:90 , srv718418587:90 , srv776290519:90 , srv790770343:90 , srv819014074:90 , srv865018006:90 , srv884354:90 }
2024-03-25 14:57:09,160 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 173) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=6914 (was 6389) - AvailableMemoryMB LEAK? -