2023-04-08 22:01:40,691 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-08 22:01:40,956 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-08 22:01:41,054 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-08 22:01:41,107 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-08 22:01:41,114 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-08 22:01:41,126 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=165, AvailableMemoryMB=4642
2023-04-08 22:01:41,127 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-08 22:01:41,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-08 22:01:41,238 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017210650:84 , srv1018060789:84 , srv106622460:84 , srv1103186141:84 , srv1134025555:84 , srv1150833373:84 , srv1167511952:84 , srv1178152536:84 , srv1219636392:84 , srv123179712:84 , srv1235887545:84 , srv1248600288:84 , srv1262589586:84 , srv1263425638:84 , srv1270750864:84 , srv1298735472:84 , srv1298913481:84 , srv1310841902:84 , srv133416199:84 , srv1334420741:84 , srv1360815365:84 , srv138780194:84 , srv1432765895:84 , srv1456197465:84 , srv150259095:84 , srv1509348617:84 , srv1510229506:84 , srv1529939225:84 , srv1561338701:84 , srv1603320906:84 , srv1616662072:84 , srv1699482735:84 , srv1730377759:84 , srv1764575940:84 , srv1785946300:84 , srv1833589411:84 , srv1849015346:84 , srv185556589:84 , srv1877047824:84 , srv1893650625:84 , srv1919137715:84 , srv1956224368:84 , srv1969568287:84 , srv2003489195:84 , srv2028537090:84 , srv2035536767:84 , srv2119603313:84 , srv2124187170:84 , srv2125994314:84 , srv2140227611:84 , srv26568270:84 , srv278454393:84 , srv287719184:84 , srv312390651:84 , srv316731584:84 , srv326412627:84 , srv337857625:84 , srv348965282:84 , srv360812510:84 , srv406680316:84 , srv413103719:84 , srv42663199:84 , srv430173989:84 , srv430703358:84 , srv43415521:84 , srv45936300:84 , srv473878881:84 , srv494483075:84 , srv496961296:84 , srv499080852:84 , srv508516945:84 , srv526844422:84 , srv555076656:84 , srv564656257:84 , srv567990408:84 , srv569052452:84 , srv576849988:84 , srv579754168:84 , srv58756119:84 , srv601390837:84 , srv630643400:84 , srv636772078:84 , srv63774833:84 , srv644549692:84 , srv665417848:84 , srv723593360:84 , srv745861337:84 , srv771009075:84 , srv774566175:84 , srv793164258:84 , srv815160148:84 , srv820486613:84 , srv876813267:84 , srv897743448:84 , srv905584140:84 , srv912070868:84 , srv926752392:84 , srv979652066:84 , srv985903402:84 , srv106201610:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-08 22:01:41,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:41,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:41,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:41,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:41,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:41,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:41,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:41,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:41,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:41,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:41,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:41,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:41,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:41,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:41,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:41,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:41,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:41,579 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-08 22:01:41,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-08 22:01:41,980 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 629 ms to try 5000 different iterations.  Found a solution that moves 4704 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1808374569124397. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849390563023788, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0502585905684384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5226666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849390563023788, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:41,995 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1270750864:81 , srv1730377759:82 , srv185556589:82 , srv1103186141:83 , srv1178152536:83 , srv1510229506:83 , srv287719184:83 , srv413103719:83 , srv508516945:83 , srv745861337:83 , srv1150833373:84 , srv1235887545:84 , srv1334420741:84 , srv1529939225:84 , srv1919137715:84 , srv430173989:84 , srv771009075:84 , srv912070868:84 , srv926752392:84 , srv123179712:85 , srv1298735472:85 , srv138780194:85 , srv1764575940:85 , srv1833589411:85 , srv1969568287:85 , srv2003489195:85 , srv316731584:85 , srv42663199:85 , srv430703358:85 , srv43415521:85 , srv555076656:85 , srv630643400:85 , srv774566175:85 , srv793164258:85 , srv106622460:86 , srv1360815365:86 , srv1432765895:86 , srv1456197465:86 , srv1616662072:86 , srv1785946300:86 , srv1849015346:86 , srv1893650625:86 , srv1956224368:86 , srv2028537090:86 , srv2035536767:86 , srv337857625:86 , srv526844422:86 , srv564656257:86 , srv576849988:86 , srv601390837:86 , srv644549692:86 , srv815160148:86 , srv820486613:86 , srv876813267:86 , srv985903402:86 , srv1017210650:87 , srv1018060789:87 , srv1167511952:87 , srv1248600288:87 , srv1263425638:87 , srv150259095:87 , srv1603320906:87 , srv1877047824:87 , srv2124187170:87 , srv26568270:87 , srv312390651:87 , srv406680316:87 , srv45936300:87 , srv494483075:87 , srv567990408:87 , srv569052452:87 , srv63774833:87 , srv1134025555:88 , srv1262589586:88 , srv1298913481:88 , srv1561338701:88 , srv1699482735:88 , srv2119603313:88 , srv360812510:88 , srv496961296:88 , srv499080852:88 , srv723593360:88 , srv905584140:88 , srv1219636392:89 , srv133416199:89 , srv58756119:89 , srv636772078:89 , srv665417848:89 , srv979652066:90 , srv1310841902:91 , srv278454393:91 , srv326412627:91 , srv897743448:91 , srv2125994314:92 , srv473878881:92 , srv1509348617:93 , srv579754168:93 , srv348965282:94 , srv2140227611:95 , srv106201610:432 }
2023-04-08 22:01:42,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:42,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:42,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:42,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:42,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:42,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:42,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:42,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:42,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:42,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:42,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:42,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:42,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849390563023788, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0502585905684384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849390563023788, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,132 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-08 22:01:42,133 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18080451880043932, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03849390563023788, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0502585905684384, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03849390563023788, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17575, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24625, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-08 22:01:42,344 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 300 ms to try 5000 different iterations.  Found a solution that moves 2670 regions; Going from a computed imbalance of 0.18080451880043932 to a new imbalance of 0.014476522187492052. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021947232345034613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033380438097589143, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2966666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021947232345034613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,353 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2140227611:77 , srv1509348617:78 , srv579754168:79 , srv348965282:80 , srv1699482735:81 , srv473878881:81 , srv1219636392:82 , srv1310841902:82 , srv1561338701:82 , srv2125994314:82 , srv360812510:82 , srv406680316:82 , srv979652066:82 , srv1017210650:83 , srv1134025555:83 , srv1298913481:83 , srv1877047824:83 , srv278454393:83 , srv636772078:83 , srv665417848:83 , srv897743448:83 , srv1167511952:84 , srv1262589586:84 , srv150259095:84 , srv1603320906:84 , srv1956224368:84 , srv2119603313:84 , srv2124187170:84 , srv326412627:84 , srv337857625:84 , srv45936300:84 , srv494483075:84 , srv569052452:84 , srv576849988:84 , srv63774833:84 , srv1248600288:85 , srv133416199:85 , srv1360815365:85 , srv1456197465:85 , srv1785946300:85 , srv1893650625:85 , srv430703358:85 , srv496961296:85 , srv499080852:85 , srv526844422:85 , srv555076656:85 , srv564656257:85 , srv58756119:85 , srv723593360:85 , srv820486613:85 , srv1018060789:86 , srv1263425638:86 , srv138780194:86 , srv2003489195:86 , srv2028537090:86 , srv2035536767:86 , srv312390651:86 , srv567990408:86 , srv601390837:86 , srv644549692:86 , srv985903402:86 , srv1432765895:87 , srv1529939225:87 , srv1616662072:87 , srv1833589411:87 , srv185556589:87 , srv1919137715:87 , srv1969568287:87 , srv316731584:87 , srv42663199:87 , srv745861337:87 , srv793164258:87 , srv876813267:87 , srv106622460:88 , srv1150833373:88 , srv1298735472:88 , srv1334420741:88 , srv1764575940:88 , srv1849015346:88 , srv26568270:88 , srv413103719:88 , srv774566175:88 , srv815160148:88 , srv905584140:88 , srv912070868:88 , srv926752392:88 , srv1178152536:89 , srv123179712:89 , srv1510229506:89 , srv1730377759:89 , srv287719184:89 , srv43415521:89 , srv508516945:89 , srv771009075:89 , srv1103186141:90 , srv1235887545:90 , srv1270750864:90 , srv430173989:91 , srv630643400:91 , srv106201610:537 }
2023-04-08 22:01:42,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:42,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:42,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:42,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:42,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:42,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:42,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:42,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:42,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021947232345034613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033380438097589143, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021947232345034613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,467 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-08 22:01:42,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014457826447899998, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021947232345034613); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033380438097589143, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021947232345034613); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-08 22:01:42,627 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 233 ms to try 5000 different iterations.  Found a solution that moves 1030 regions; Going from a computed imbalance of 0.014457826447899998 to a new imbalance of 6.876084305461971E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006765079824034915); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008914105645620285); MoveCostFunction : (multiplier=7.0, imbalance=0.11444444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006765079824034915); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,628 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv26568270:82 , srv430173989:82 , srv630643400:82 , srv905584140:82 , srv123179712:83 , srv1310841902:83 , srv1849015346:83 , srv2125994314:83 , srv278454393:83 , srv326412627:83 , srv348965282:83 , srv43415521:83 , srv58756119:83 , srv771009075:83 , srv106622460:84 , srv1235887545:84 , srv1263425638:84 , srv1298735472:84 , srv133416199:84 , srv1432765895:84 , srv1764575940:84 , srv473878881:84 , srv496961296:84 , srv499080852:84 , srv723593360:84 , srv774566175:84 , srv815160148:84 , srv897743448:84 , srv1018060789:85 , srv1103186141:85 , srv1150833373:85 , srv1248600288:85 , srv1262589586:85 , srv1334420741:85 , srv1510229506:85 , srv1616662072:85 , srv2028537090:85 , srv2035536767:85 , srv2140227611:85 , srv287719184:85 , srv312390651:85 , srv42663199:85 , srv567990408:85 , srv579754168:85 , srv636772078:85 , srv644549692:85 , srv793164258:85 , srv876813267:85 , srv912070868:85 , srv926752392:85 , srv985903402:85 , srv1134025555:86 , srv1178152536:86 , srv1219636392:86 , srv1270750864:86 , srv1298913481:86 , srv1360815365:86 , srv138780194:86 , srv150259095:86 , srv1529939225:86 , srv1730377759:86 , srv1833589411:86 , srv1893650625:86 , srv1919137715:86 , srv2119603313:86 , srv2124187170:86 , srv316731584:86 , srv45936300:86 , srv494483075:86 , srv508516945:86 , srv569052452:86 , srv601390837:86 , srv665417848:86 , srv820486613:86 , srv979652066:86 , srv1017210650:87 , srv1167511952:87 , srv1456197465:87 , srv1509348617:87 , srv1561338701:87 , srv1603320906:87 , srv1877047824:87 , srv1956224368:87 , srv1969568287:87 , srv2003489195:87 , srv360812510:87 , srv413103719:87 , srv526844422:87 , srv63774833:87 , srv745861337:87 , srv1699482735:88 , srv1785946300:88 , srv337857625:88 , srv406680316:88 , srv430703358:88 , srv555076656:88 , srv564656257:88 , srv576849988:88 , srv185556589:89 , srv106201610:549 }
2023-04-08 22:01:42,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:42,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:42,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:42,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:42,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:42,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:42,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:42,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:42,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:42,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:42,726 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006765079824034915); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008914105645620285); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006765079824034915); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,726 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-08 22:01:42,726 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.803962163964532E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006765079824034915); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008914105645620285); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006765079824034915); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-08 22:01:42,945 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 292 ms to try 5000 different iterations.  Found a solution that moves 235 regions; Going from a computed imbalance of 6.803962163964532E-4 to a new imbalance of 2.7462707370392156E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.026111111111111113, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:42,946 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1603320906:83 , srv1969568287:83 , srv312390651:83 , srv564656257:83 , srv665417848:83 , srv1018060789:84 , srv106622460:84 , srv1134025555:84 , srv1167511952:84 , srv1178152536:84 , srv133416199:84 , srv1456197465:84 , srv1509348617:84 , srv1785946300:84 , srv1833589411:84 , srv185556589:84 , srv2003489195:84 , srv2035536767:84 , srv2125994314:84 , srv316731584:84 , srv326412627:84 , srv337857625:84 , srv413103719:84 , srv45936300:84 , srv508516945:84 , srv526844422:84 , srv555076656:84 , srv567990408:84 , srv576849988:84 , srv630643400:84 , srv63774833:84 , srv793164258:84 , srv815160148:84 , srv876813267:84 , srv897743448:84 , srv905584140:84 , srv979652066:84 , srv1017210650:85 , srv1103186141:85 , srv1150833373:85 , srv1219636392:85 , srv123179712:85 , srv1235887545:85 , srv1248600288:85 , srv1262589586:85 , srv1263425638:85 , srv1270750864:85 , srv1298735472:85 , srv1298913481:85 , srv1310841902:85 , srv1334420741:85 , srv1360815365:85 , srv138780194:85 , srv1432765895:85 , srv150259095:85 , srv1510229506:85 , srv1561338701:85 , srv1616662072:85 , srv1699482735:85 , srv1730377759:85 , srv1764575940:85 , srv1849015346:85 , srv1877047824:85 , srv1893650625:85 , srv1919137715:85 , srv1956224368:85 , srv2028537090:85 , srv2119603313:85 , srv2124187170:85 , srv2140227611:85 , srv26568270:85 , srv278454393:85 , srv287719184:85 , srv348965282:85 , srv360812510:85 , srv406680316:85 , srv42663199:85 , srv430173989:85 , srv430703358:85 , srv43415521:85 , srv473878881:85 , srv494483075:85 , srv496961296:85 , srv499080852:85 , srv569052452:85 , srv579754168:85 , srv58756119:85 , srv601390837:85 , srv636772078:85 , srv644549692:85 , srv723593360:85 , srv745861337:85 , srv774566175:85 , srv820486613:85 , srv926752392:85 , srv985903402:85 , srv1529939225:86 , srv771009075:86 , srv912070868:86 , srv106201610:624 }
2023-04-08 22:01:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:42,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:42,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:42,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:42,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:42,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:42,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:42,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:42,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:43,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:43,041 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-08 22:01:43,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.581720220030494E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-08 22:01:43,188 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 217 ms to try 5000 different iterations.  Found a solution that moves 29 regions; Going from a computed imbalance of 2.581720220030494E-5 to a new imbalance of 6.601048012408121E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0032222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:43,188 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1103186141:83 , srv1235887545:83 , srv1529939225:83 , srv1616662072:83 , srv1785946300:83 , srv2119603313:83 , srv430703358:83 , srv601390837:83 , srv912070868:83 , srv1017210650:84 , srv1018060789:84 , srv106622460:84 , srv1150833373:84 , srv1167511952:84 , srv1178152536:84 , srv1219636392:84 , srv123179712:84 , srv1248600288:84 , srv1262589586:84 , srv1263425638:84 , srv1270750864:84 , srv1298735472:84 , srv1298913481:84 , srv133416199:84 , srv1334420741:84 , srv1360815365:84 , srv138780194:84 , srv1432765895:84 , srv1456197465:84 , srv150259095:84 , srv1509348617:84 , srv1510229506:84 , srv1561338701:84 , srv1699482735:84 , srv1730377759:84 , srv1764575940:84 , srv1833589411:84 , srv1849015346:84 , srv185556589:84 , srv1877047824:84 , srv1893650625:84 , srv1919137715:84 , srv1956224368:84 , srv1969568287:84 , srv2003489195:84 , srv2028537090:84 , srv2124187170:84 , srv2140227611:84 , srv26568270:84 , srv278454393:84 , srv287719184:84 , srv316731584:84 , srv326412627:84 , srv337857625:84 , srv348965282:84 , srv360812510:84 , srv406680316:84 , srv413103719:84 , srv42663199:84 , srv430173989:84 , srv43415521:84 , srv473878881:84 , srv494483075:84 , srv496961296:84 , srv499080852:84 , srv508516945:84 , srv526844422:84 , srv555076656:84 , srv564656257:84 , srv567990408:84 , srv569052452:84 , srv576849988:84 , srv579754168:84 , srv58756119:84 , srv630643400:84 , srv636772078:84 , srv63774833:84 , srv644549692:84 , srv723593360:84 , srv745861337:84 , srv771009075:84 , srv774566175:84 , srv815160148:84 , srv820486613:84 , srv876813267:84 , srv897743448:84 , srv905584140:84 , srv926752392:84 , srv979652066:84 , srv985903402:84 , srv1134025555:85 , srv1310841902:85 , srv1603320906:85 , srv2035536767:85 , srv2125994314:85 , srv312390651:85 , srv45936300:85 , srv665417848:85 , srv793164258:85 , srv106201610:684 }
2023-04-08 22:01:43,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:43,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167511952=7, srv348965282=58, srv1956224368=42, srv912070868=96, srv499080852=70, srv1235887545=11, srv43415521=65, srv185556589=38, srv150259095=25, srv430703358=64, srv1603320906=30, srv897743448=94, srv1270750864=15, srv473878881=67, srv138780194=22, srv1017210650=0, srv1298913481=17, srv1877047824=39, srv508516945=71, srv1509348617=26, srv1919137715=41, srv1018060789=1, srv494483075=68, srv1699482735=32, srv133416199=19, srv601390837=80, srv2124187170=48, srv278454393=52, srv1103186141=4, srv360812510=59, srv42663199=62, srv567990408=75, srv1263425638=14, srv337857625=57, srv1849015346=37, srv2035536767=46, srv876813267=93, srv413103719=61, srv555076656=73, srv2003489195=44, srv665417848=85, srv644549692=84, srv1178152536=8, srv1561338701=29, srv1219636392=9, srv312390651=54, srv579754168=78, srv496961296=69, srv1334420741=20, srv926752392=97, srv905584140=95, srv2125994314=49, srv1310841902=18, srv106622460=3, srv1969568287=43, srv106201610=2, srv58756119=79, srv1893650625=40, srv1510229506=27, srv636772078=82, srv630643400=81, srv2140227611=50, srv1529939225=28, srv1833589411=36, srv1150833373=6, srv26568270=51, srv1764575940=34, srv526844422=72, srv123179712=10, srv1730377759=33, srv985903402=99, srv406680316=60, srv2028537090=45, srv1262589586=13, srv1432765895=23, srv45936300=66, srv820486613=92, srv979652066=98, srv815160148=91, srv2119603313=47, srv1134025555=5, srv1248600288=12, srv569052452=76, srv326412627=56, srv1616662072=31, srv745861337=87, srv1298735472=16, srv1456197465=24, srv793164258=90, srv771009075=88, srv723593360=86, srv63774833=83, srv774566175=89, srv430173989=63, srv564656257=74, srv1360815365=21, srv287719184=53, srv316731584=55, srv1785946300=35, srv576849988=77} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:43,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:43,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-08 22:01:43,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-08 22:01:43,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-08 22:01:43,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-08 22:01:43,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-08 22:01:43,284 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:43,284 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1103186141:83 , srv1235887545:83 , srv1529939225:83 , srv1616662072:83 , srv1785946300:83 , srv2119603313:83 , srv430703358:83 , srv601390837:83 , srv912070868:83 , srv1017210650:84 , srv1018060789:84 , srv106622460:84 , srv1150833373:84 , srv1167511952:84 , srv1178152536:84 , srv1219636392:84 , srv123179712:84 , srv1248600288:84 , srv1262589586:84 , srv1263425638:84 , srv1270750864:84 , srv1298735472:84 , srv1298913481:84 , srv133416199:84 , srv1334420741:84 , srv1360815365:84 , srv138780194:84 , srv1432765895:84 , srv1456197465:84 , srv150259095:84 , srv1509348617:84 , srv1510229506:84 , srv1561338701:84 , srv1699482735:84 , srv1730377759:84 , srv1764575940:84 , srv1833589411:84 , srv1849015346:84 , srv185556589:84 , srv1877047824:84 , srv1893650625:84 , srv1919137715:84 , srv1956224368:84 , srv1969568287:84 , srv2003489195:84 , srv2028537090:84 , srv2124187170:84 , srv2140227611:84 , srv26568270:84 , srv278454393:84 , srv287719184:84 , srv316731584:84 , srv326412627:84 , srv337857625:84 , srv348965282:84 , srv360812510:84 , srv406680316:84 , srv413103719:84 , srv42663199:84 , srv430173989:84 , srv43415521:84 , srv473878881:84 , srv494483075:84 , srv496961296:84 , srv499080852:84 , srv508516945:84 , srv526844422:84 , srv555076656:84 , srv564656257:84 , srv567990408:84 , srv569052452:84 , srv576849988:84 , srv579754168:84 , srv58756119:84 , srv630643400:84 , srv636772078:84 , srv63774833:84 , srv644549692:84 , srv723593360:84 , srv745861337:84 , srv771009075:84 , srv774566175:84 , srv815160148:84 , srv820486613:84 , srv876813267:84 , srv897743448:84 , srv905584140:84 , srv926752392:84 , srv979652066:84 , srv985903402:84 , srv1134025555:85 , srv1310841902:85 , srv1603320906:85 , srv2035536767:85 , srv2125994314:85 , srv312390651:85 , srv45936300:85 , srv665417848:85 , srv793164258:85 , srv106201610:684 }
2023-04-08 22:01:43,306 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=263 (was 260) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 165), AvailableMemoryMB=4149 (was 4642)
2023-04-08 22:01:43,312 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=263, ProcessCount=165, AvailableMemoryMB=4148
2023-04-08 22:01:43,313 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-08 22:01:43,313 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-08 22:01:43,320 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1039595567:84 , srv1077766352:84 , srv1083196917:84 , srv1113017259:84 , srv116872896:84 , srv1268584475:84 , srv1270687818:84 , srv1275148418:84 , srv1329645833:84 , srv1399378890:84 , srv1728664256:84 , srv1728683258:84 , srv1767927691:84 , srv1773442684:84 , srv1777994750:84 , srv2008320477:84 , srv2020400570:84 , srv2059700430:84 , srv220818528:84 , srv421319409:84 , srv514865390:84 , srv656533947:84 , srv670668140:84 , srv677729203:84 , srv759458300:84 , srv812443398:84 , srv820493702:84 , srv892791073:84 , srv896348603:84 , srv1571935219:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-08 22:01:43,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:01:43,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1777994750=15, srv2008320477=16, srv896348603=29, srv220818528=19, srv1329645833=8, srv677729203=24, srv1113017259=3, srv1039595567=0, srv116872896=4, srv1275148418=7, srv1268584475=5, srv1270687818=6, srv1399378890=9, srv759458300=25, srv2059700430=18, srv1767927691=13, srv421319409=20, srv514865390=21, srv1083196917=2, srv670668140=23, srv820493702=27, srv1077766352=1, srv2020400570=17, srv1728683258=12, srv1773442684=14, srv1571935219=10, srv1728664256=11, srv892791073=28, srv656533947=22, srv812443398=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:01:43,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:01:43,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:01:43,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:01:43,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:01:43,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:01:43,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:01:43,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:01:43,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:01:43,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-08 22:01:43,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.133359936596962, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:01:43,347 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912613256069894, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.133359936596962, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-08 22:03:43,335 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9377641 different iterations.  Found a solution that moves 1826 regions; Going from a computed imbalance of 0.9912613256069894 to a new imbalance of 4.2619750930202236E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6762962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:03:43,340 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1039595567:90 , srv1077766352:90 , srv1083196917:90 , srv1113017259:90 , srv116872896:90 , srv1268584475:90 , srv1270687818:90 , srv1275148418:90 , srv1329645833:90 , srv1399378890:90 , srv1571935219:90 , srv1728664256:90 , srv1728683258:90 , srv1767927691:90 , srv1773442684:90 , srv1777994750:90 , srv2008320477:90 , srv2020400570:90 , srv2059700430:90 , srv220818528:90 , srv421319409:90 , srv514865390:90 , srv656533947:90 , srv670668140:90 , srv677729203:90 , srv759458300:90 , srv812443398:90 , srv820493702:90 , srv892791073:90 , srv896348603:90 }
2023-04-08 22:03:43,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-08 22:03:43,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1777994750=15, srv2008320477=16, srv896348603=29, srv220818528=19, srv1329645833=8, srv677729203=24, srv1113017259=3, srv1039595567=0, srv116872896=4, srv1275148418=7, srv1268584475=5, srv1270687818=6, srv1399378890=9, srv759458300=25, srv2059700430=18, srv1767927691=13, srv421319409=20, srv514865390=21, srv1083196917=2, srv670668140=23, srv820493702=27, srv1077766352=1, srv2020400570=17, srv1728683258=12, srv1773442684=14, srv1571935219=10, srv1728664256=11, srv892791073=28, srv656533947=22, srv812443398=26} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-08 22:03:43,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-08 22:03:43,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-08 22:03:43,357 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-08 22:03:43,358 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1039595567:90 , srv1077766352:90 , srv1083196917:90 , srv1113017259:90 , srv116872896:90 , srv1268584475:90 , srv1270687818:90 , srv1275148418:90 , srv1329645833:90 , srv1399378890:90 , srv1571935219:90 , srv1728664256:90 , srv1728683258:90 , srv1767927691:90 , srv1773442684:90 , srv1777994750:90 , srv2008320477:90 , srv2020400570:90 , srv2059700430:90 , srv220818528:90 , srv421319409:90 , srv514865390:90 , srv656533947:90 , srv670668140:90 , srv677729203:90 , srv759458300:90 , srv812443398:90 , srv820493702:90 , srv892791073:90 , srv896348603:90 }
2023-04-08 22:03:43,372 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 263), ProcessCount=165 (was 165), AvailableMemoryMB=4161 (was 4148) - AvailableMemoryMB LEAK? -