2023-07-20 23:53:11,807 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-20 23:53:12,119 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-20 23:53:12,230 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-20 23:53:12,306 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-20 23:53:12,316 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-20 23:53:12,334 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=236, ProcessCount=175, AvailableMemoryMB=7305
2023-07-20 23:53:12,337 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-20 23:53:12,337 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-20 23:53:12,485 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1023294053:84 , srv1077228892:84 , srv1089536298:84 , srv1100411498:84 , srv1102921649:84 , srv1122143099:84 , srv1148741432:84 , srv1157752796:84 , srv1170126826:84 , srv1190969562:84 , srv1191921748:84 , srv1203418774:84 , srv1300535055:84 , srv1321115189:84 , srv1329630296:84 , srv1344794738:84 , srv1353621466:84 , srv1399250787:84 , srv1423566698:84 , srv1424925657:84 , srv1430148303:84 , srv1430711229:84 , srv148713871:84 , srv1516847023:84 , srv1520074523:84 , srv1522317246:84 , srv1527579531:84 , srv1545754491:84 , srv1551028784:84 , srv1558723296:84 , srv1559974196:84 , srv1575036779:84 , srv1587458085:84 , srv1590326780:84 , srv1591418015:84 , srv1648478336:84 , srv1654165261:84 , srv1657349747:84 , srv1668067104:84 , srv1705186522:84 , srv1732000624:84 , srv1755256991:84 , srv1771066878:84 , srv1802595984:84 , srv1809521080:84 , srv1816688130:84 , srv1819854468:84 , srv186848820:84 , srv188722900:84 , srv1899144323:84 , srv1978795958:84 , srv1996302135:84 , srv1996562873:84 , srv2031243813:84 , srv2044963543:84 , srv2056036135:84 , srv2091887196:84 , srv2097239620:84 , srv2117887408:84 , srv2120000703:84 , srv230083244:84 , srv242778002:84 , srv248794390:84 , srv276654808:84 , srv326017317:84 , srv349534589:84 , srv395101174:84 , srv423245132:84 , srv436689620:84 , srv464636352:84 , srv468877185:84 , srv469354977:84 , srv47352519:84 , srv508679844:84 , srv551973874:84 , srv553610181:84 , srv59158733:84 , srv642353784:84 , srv643923397:84 , srv650570137:84 , srv689277775:84 , srv693523911:84 , srv721368308:84 , srv721837296:84 , srv728835611:84 , srv744762150:84 , srv745787029:84 , srv756470320:84 , srv761812666:84 , srv764024196:84 , srv768420101:84 , srv796930096:84 , srv853737336:84 , srv882883298:84 , srv914156470:84 , srv923512796:84 , srv925485839:84 , srv928935553:84 , srv984473573:84 , srv535048671:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-20 23:53:12,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:12,511 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:12,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:12,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:12,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:12,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:12,928 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:12,928 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-20 23:53:12,928 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-20 23:53:13,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 639 ms to try 5000 different iterations.  Found a solution that moves 4705 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17781942634654668. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04173772466913091, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05798525297254089, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5227777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04173772466913091, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23883333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:13,289 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv745787029:81 , srv1516847023:82 , srv925485839:82 , srv1527579531:83 , srv1802595984:83 , srv188722900:83 , srv1899144323:83 , srv1978795958:83 , srv2097239620:83 , srv230083244:83 , srv423245132:83 , srv508679844:83 , srv1157752796:84 , srv1191921748:84 , srv1344794738:84 , srv1545754491:84 , srv1559974196:84 , srv1657349747:84 , srv1732000624:84 , srv186848820:84 , srv1996562873:84 , srv2117887408:84 , srv326017317:84 , srv47352519:84 , srv59158733:84 , srv689277775:84 , srv721368308:84 , srv721837296:84 , srv796930096:84 , srv923512796:84 , srv984473573:84 , srv1100411498:85 , srv1122143099:85 , srv1300535055:85 , srv1399250787:85 , srv1430148303:85 , srv1575036779:85 , srv1648478336:85 , srv2056036135:85 , srv242778002:85 , srv248794390:85 , srv551973874:85 , srv914156470:85 , srv1023294053:86 , srv1102921649:86 , srv1170126826:86 , srv1203418774:86 , srv1321115189:86 , srv1353621466:86 , srv1590326780:86 , srv1591418015:86 , srv1668067104:86 , srv2120000703:86 , srv349534589:86 , srv468877185:86 , srv553610181:86 , srv728835611:86 , srv761812666:86 , srv1077228892:87 , srv1190969562:87 , srv1329630296:87 , srv1424925657:87 , srv1430711229:87 , srv148713871:87 , srv1558723296:87 , srv1587458085:87 , srv436689620:87 , srv744762150:87 , srv882883298:87 , srv928935553:87 , srv1423566698:88 , srv1520074523:88 , srv1522317246:88 , srv1771066878:88 , srv1816688130:88 , srv2044963543:88 , srv2091887196:88 , srv469354977:88 , srv643923397:88 , srv1996302135:89 , srv2031243813:89 , srv464636352:89 , srv642353784:89 , srv764024196:89 , srv768420101:89 , srv1551028784:90 , srv1654165261:90 , srv1705186522:90 , srv1809521080:90 , srv276654808:90 , srv395101174:90 , srv650570137:90 , srv756470320:90 , srv1089536298:91 , srv1755256991:91 , srv693523911:91 , srv853737336:91 , srv1148741432:93 , srv1819854468:94 , srv535048671:461 }
2023-07-20 23:53:13,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:13,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:13,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:13,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:13,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:13,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:13,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:13,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:13,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:13,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:13,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04173772466913091, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05798525297254089, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04173772466913091, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23883333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:13,464 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-20 23:53:13,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17778648123239663, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04173772466913091, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05798525297254089, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04173772466913091, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17308333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23883333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-20 23:53:13,788 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 449 ms to try 5000 different iterations.  Found a solution that moves 2584 regions; Going from a computed imbalance of 0.17778648123239663 to a new imbalance of 0.014353261693239834. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0232827735050518); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03303570832737446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2871111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0232827735050518); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:13,811 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1819854468:78 , srv1089536298:80 , srv1551028784:80 , srv276654808:80 , srv1148741432:81 , srv756470320:81 , srv1809521080:82 , srv1996302135:82 , srv395101174:82 , srv642353784:82 , srv853737336:82 , srv1423566698:83 , srv148713871:83 , srv1520074523:83 , srv1522317246:83 , srv1558723296:83 , srv1705186522:83 , srv1755256991:83 , srv1771066878:83 , srv1816688130:83 , srv2044963543:83 , srv436689620:83 , srv643923397:83 , srv650570137:83 , srv764024196:83 , srv928935553:83 , srv1424925657:84 , srv2091887196:84 , srv464636352:84 , srv693523911:84 , srv744762150:84 , srv1023294053:85 , srv1190969562:85 , srv1321115189:85 , srv1430711229:85 , srv1575036779:85 , srv1591418015:85 , srv768420101:85 , srv914156470:85 , srv1077228892:86 , srv1100411498:86 , srv1102921649:86 , srv1122143099:86 , srv1157752796:86 , srv1203418774:86 , srv1329630296:86 , srv1353621466:86 , srv1430148303:86 , srv1587458085:86 , srv1654165261:86 , srv2031243813:86 , srv2120000703:86 , srv248794390:86 , srv728835611:86 , srv761812666:86 , srv882883298:86 , srv984473573:86 , srv1170126826:87 , srv1191921748:87 , srv1300535055:87 , srv1545754491:87 , srv1590326780:87 , srv1732000624:87 , srv186848820:87 , srv188722900:87 , srv1996562873:87 , srv2056036135:87 , srv242778002:87 , srv468877185:87 , srv469354977:87 , srv47352519:87 , srv551973874:87 , srv553610181:87 , srv689277775:87 , srv721368308:87 , srv923512796:87 , srv1399250787:88 , srv1527579531:88 , srv1668067104:88 , srv1802595984:88 , srv2097239620:88 , srv2117887408:88 , srv326017317:88 , srv349534589:88 , srv796930096:88 , srv925485839:88 , srv1559974196:89 , srv1648478336:89 , srv1657349747:89 , srv1899144323:89 , srv1978795958:89 , srv230083244:89 , srv423245132:89 , srv508679844:89 , srv59158733:89 , srv721837296:89 , srv1516847023:90 , srv745787029:91 , srv1344794738:93 , srv535048671:520 }
2023-07-20 23:53:13,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:13,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:13,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:13,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:13,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:13,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:13,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:13,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:13,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:13,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:13,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:13,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:13,980 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0232827735050518); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03303570832737446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0232827735050518); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:13,980 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-20 23:53:13,981 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014335168138518536, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0232827735050518); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03303570832737446, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0232827735050518); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-20 23:53:14,373 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 507 ms to try 5000 different iterations.  Found a solution that moves 990 regions; Going from a computed imbalance of 0.014335168138518536 to a new imbalance of 7.020286122383238E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00823340762517771); MoveCostFunction : (multiplier=7.0, imbalance=0.11, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:14,374 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1654165261:81 , srv2031243813:82 , srv469354977:82 , srv693523911:82 , srv1148741432:83 , srv1344794738:83 , srv1668067104:83 , srv349534589:83 , srv721837296:83 , srv768420101:83 , srv1170126826:84 , srv1399250787:84 , srv1559974196:84 , srv1587458085:84 , srv1648478336:84 , srv1705186522:84 , srv1755256991:84 , srv650570137:84 , srv761812666:84 , srv1077228892:85 , srv1100411498:85 , srv1102921649:85 , srv1190969562:85 , srv1300535055:85 , srv1329630296:85 , srv1353621466:85 , srv1590326780:85 , srv1591418015:85 , srv1809521080:85 , srv1819854468:85 , srv1978795958:85 , srv2056036135:85 , srv2097239620:85 , srv2117887408:85 , srv395101174:85 , srv423245132:85 , srv464636352:85 , srv508679844:85 , srv551973874:85 , srv553610181:85 , srv59158733:85 , srv721368308:85 , srv764024196:85 , srv796930096:85 , srv853737336:85 , srv882883298:85 , srv923512796:85 , srv1089536298:86 , srv1191921748:86 , srv1203418774:86 , srv1321115189:86 , srv1423566698:86 , srv1424925657:86 , srv1516847023:86 , srv1520074523:86 , srv1522317246:86 , srv1545754491:86 , srv1657349747:86 , srv1771066878:86 , srv1802595984:86 , srv186848820:86 , srv1899144323:86 , srv1996562873:86 , srv2044963543:86 , srv2091887196:86 , srv2120000703:86 , srv230083244:86 , srv242778002:86 , srv248794390:86 , srv326017317:86 , srv468877185:86 , srv47352519:86 , srv642353784:86 , srv643923397:86 , srv689277775:86 , srv728835611:86 , srv744762150:86 , srv745787029:86 , srv1023294053:87 , srv1122143099:87 , srv1157752796:87 , srv1430148303:87 , srv1430711229:87 , srv1527579531:87 , srv1551028784:87 , srv1575036779:87 , srv1732000624:87 , srv188722900:87 , srv1996302135:87 , srv276654808:87 , srv756470320:87 , srv914156470:87 , srv925485839:87 , srv928935553:87 , srv984473573:87 , srv148713871:88 , srv1558723296:88 , srv1816688130:88 , srv436689620:88 , srv535048671:542 }
2023-07-20 23:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:14,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:14,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:14,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:14,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:14,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:14,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:14,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:14,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:14,529 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00823340762517771); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:14,529 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-20 23:53:14,530 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.950964840749777E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007338059219328525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00823340762517771); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007338059219328525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-20 23:53:14,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 479 ms to try 5000 different iterations.  Found a solution that moves 292 regions; Going from a computed imbalance of 6.950964840749777E-4 to a new imbalance of 5.673900084292141E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.03244444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:14,898 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1344794738:82 , srv1430148303:83 , srv1430711229:83 , srv1657349747:83 , srv1816688130:83 , srv468877185:83 , srv882883298:83 , srv1023294053:84 , srv1077228892:84 , srv1122143099:84 , srv1157752796:84 , srv1203418774:84 , srv1329630296:84 , srv148713871:84 , srv1527579531:84 , srv1558723296:84 , srv1559974196:84 , srv1590326780:84 , srv1648478336:84 , srv1732000624:84 , srv1755256991:84 , srv1996302135:84 , srv2091887196:84 , srv2120000703:84 , srv230083244:84 , srv242778002:84 , srv326017317:84 , srv464636352:84 , srv553610181:84 , srv59158733:84 , srv728835611:84 , srv745787029:84 , srv756470320:84 , srv853737336:84 , srv1089536298:85 , srv1102921649:85 , srv1148741432:85 , srv1170126826:85 , srv1190969562:85 , srv1191921748:85 , srv1300535055:85 , srv1321115189:85 , srv1353621466:85 , srv1399250787:85 , srv1423566698:85 , srv1424925657:85 , srv1516847023:85 , srv1520074523:85 , srv1522317246:85 , srv1545754491:85 , srv1551028784:85 , srv1575036779:85 , srv1587458085:85 , srv1591418015:85 , srv1654165261:85 , srv1668067104:85 , srv1705186522:85 , srv1802595984:85 , srv1809521080:85 , srv1819854468:85 , srv186848820:85 , srv188722900:85 , srv1899144323:85 , srv1978795958:85 , srv1996562873:85 , srv2031243813:85 , srv2044963543:85 , srv2056036135:85 , srv2117887408:85 , srv248794390:85 , srv276654808:85 , srv349534589:85 , srv395101174:85 , srv423245132:85 , srv436689620:85 , srv469354977:85 , srv47352519:85 , srv508679844:85 , srv551973874:85 , srv642353784:85 , srv643923397:85 , srv650570137:85 , srv689277775:85 , srv693523911:85 , srv764024196:85 , srv768420101:85 , srv796930096:85 , srv914156470:85 , srv925485839:85 , srv928935553:85 , srv984473573:85 , srv1100411498:86 , srv1771066878:86 , srv2097239620:86 , srv721368308:86 , srv721837296:86 , srv744762150:86 , srv761812666:86 , srv923512796:86 , srv535048671:619 }
2023-07-20 23:53:14,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:14,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:14,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:14,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:14,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:14,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:14,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:14,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:14,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:14,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:14,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:14,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:14,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:15,050 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:15,050 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-20 23:53:15,050 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.469437314221729E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-20 23:53:15,383 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 25 regions; Going from a computed imbalance of 5.469437314221729E-5 to a new imbalance of 6.919122795330589E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.002777777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:15,383 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1516847023:83 , srv1771066878:83 , srv1899144323:83 , srv436689620:83 , srv744762150:83 , srv1023294053:84 , srv1077228892:84 , srv1089536298:84 , srv1100411498:84 , srv1102921649:84 , srv1122143099:84 , srv1148741432:84 , srv1170126826:84 , srv1190969562:84 , srv1191921748:84 , srv1203418774:84 , srv1300535055:84 , srv1321115189:84 , srv1329630296:84 , srv1344794738:84 , srv1353621466:84 , srv1399250787:84 , srv1423566698:84 , srv1424925657:84 , srv1430711229:84 , srv148713871:84 , srv1520074523:84 , srv1522317246:84 , srv1527579531:84 , srv1545754491:84 , srv1551028784:84 , srv1558723296:84 , srv1575036779:84 , srv1587458085:84 , srv1590326780:84 , srv1648478336:84 , srv1654165261:84 , srv1657349747:84 , srv1668067104:84 , srv1705186522:84 , srv1732000624:84 , srv1755256991:84 , srv1802595984:84 , srv1809521080:84 , srv1816688130:84 , srv1819854468:84 , srv186848820:84 , srv188722900:84 , srv1978795958:84 , srv1996302135:84 , srv1996562873:84 , srv2031243813:84 , srv2044963543:84 , srv2056036135:84 , srv2091887196:84 , srv2097239620:84 , srv2117887408:84 , srv2120000703:84 , srv230083244:84 , srv242778002:84 , srv248794390:84 , srv276654808:84 , srv326017317:84 , srv349534589:84 , srv395101174:84 , srv423245132:84 , srv464636352:84 , srv468877185:84 , srv469354977:84 , srv47352519:84 , srv508679844:84 , srv551973874:84 , srv553610181:84 , srv59158733:84 , srv642353784:84 , srv643923397:84 , srv650570137:84 , srv689277775:84 , srv693523911:84 , srv721368308:84 , srv721837296:84 , srv728835611:84 , srv745787029:84 , srv756470320:84 , srv761812666:84 , srv764024196:84 , srv768420101:84 , srv796930096:84 , srv853737336:84 , srv914156470:84 , srv923512796:84 , srv925485839:84 , srv928935553:84 , srv984473573:84 , srv1157752796:85 , srv1430148303:85 , srv1559974196:85 , srv1591418015:85 , srv882883298:85 , srv535048671:684 }
2023-07-20 23:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv768420101=91, srv1157752796=7, srv464636352=69, srv1587458085=32, srv689277775=81, srv761812666=89, srv650570137=80, srv1102921649=4, srv1023294053=0, srv188722900=48, srv469354977=71, srv1755256991=41, srv1170126826=8, srv721368308=83, srv1819854468=46, srv2031243813=53, srv148713871=22, srv59158733=77, srv186848820=47, srv553610181=76, srv230083244=60, srv1978795958=50, srv1430148303=20, srv1089536298=2, srv882883298=94, srv1809521080=44, srv1551028784=28, srv436689620=68, srv1590326780=33, srv1899144323=49, srv984473573=99, srv1344794738=15, srv1399250787=17, srv535048671=74, srv1648478336=35, srv1203418774=11, srv1996302135=51, srv1705186522=39, srv248794390=62, srv1516847023=23, srv423245132=67, srv1802595984=43, srv1527579531=26, srv721837296=84, srv551973874=75, srv1329630296=14, srv1423566698=18, srv1771066878=42, srv1545754491=27, srv642353784=78, srv1657349747=37, srv276654808=63, srv1122143099=5, srv914156470=95, srv242778002=61, srv47352519=72, srv928935553=98, srv745787029=87, srv326017317=64, srv2091887196=56, srv744762150=86, srv1148741432=6, srv1654165261=36, srv1424925657=19, srv2097239620=57, srv1190969562=9, srv764024196=90, srv1321115189=13, srv395101174=66, srv2044963543=54, srv1559974196=30, srv1558723296=29, srv1996562873=52, srv1100411498=3, srv923512796=96, srv1668067104=38, srv2056036135=55, srv349534589=65, srv925485839=97, srv796930096=92, srv1300535055=12, srv756470320=88, srv508679844=73, srv1077228892=1, srv468877185=70, srv643923397=79, srv1591418015=34, srv1353621466=16, srv1732000624=40, srv2117887408=58, srv2120000703=59, srv728835611=85, srv693523911=82, srv1191921748=10, srv1575036779=31, srv1520074523=24, srv853737336=93, srv1816688130=45, srv1522317246=25, srv1430711229=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:15,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:15,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:15,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:15,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:15,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:15,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-20 23:53:15,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-20 23:53:15,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-20 23:53:15,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-20 23:53:15,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-20 23:53:15,533 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:15,533 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1516847023:83 , srv1771066878:83 , srv1899144323:83 , srv436689620:83 , srv744762150:83 , srv1023294053:84 , srv1077228892:84 , srv1089536298:84 , srv1100411498:84 , srv1102921649:84 , srv1122143099:84 , srv1148741432:84 , srv1170126826:84 , srv1190969562:84 , srv1191921748:84 , srv1203418774:84 , srv1300535055:84 , srv1321115189:84 , srv1329630296:84 , srv1344794738:84 , srv1353621466:84 , srv1399250787:84 , srv1423566698:84 , srv1424925657:84 , srv1430711229:84 , srv148713871:84 , srv1520074523:84 , srv1522317246:84 , srv1527579531:84 , srv1545754491:84 , srv1551028784:84 , srv1558723296:84 , srv1575036779:84 , srv1587458085:84 , srv1590326780:84 , srv1648478336:84 , srv1654165261:84 , srv1657349747:84 , srv1668067104:84 , srv1705186522:84 , srv1732000624:84 , srv1755256991:84 , srv1802595984:84 , srv1809521080:84 , srv1816688130:84 , srv1819854468:84 , srv186848820:84 , srv188722900:84 , srv1978795958:84 , srv1996302135:84 , srv1996562873:84 , srv2031243813:84 , srv2044963543:84 , srv2056036135:84 , srv2091887196:84 , srv2097239620:84 , srv2117887408:84 , srv2120000703:84 , srv230083244:84 , srv242778002:84 , srv248794390:84 , srv276654808:84 , srv326017317:84 , srv349534589:84 , srv395101174:84 , srv423245132:84 , srv464636352:84 , srv468877185:84 , srv469354977:84 , srv47352519:84 , srv508679844:84 , srv551973874:84 , srv553610181:84 , srv59158733:84 , srv642353784:84 , srv643923397:84 , srv650570137:84 , srv689277775:84 , srv693523911:84 , srv721368308:84 , srv721837296:84 , srv728835611:84 , srv745787029:84 , srv756470320:84 , srv761812666:84 , srv764024196:84 , srv768420101:84 , srv796930096:84 , srv853737336:84 , srv914156470:84 , srv923512796:84 , srv925485839:84 , srv928935553:84 , srv984473573:84 , srv1157752796:85 , srv1430148303:85 , srv1559974196:85 , srv1591418015:85 , srv882883298:85 , srv535048671:684 }
2023-07-20 23:53:15,566 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=265 (was 236) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=6440 (was 7305)
2023-07-20 23:53:15,578 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=265, ProcessCount=175, AvailableMemoryMB=6439
2023-07-20 23:53:15,579 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-20 23:53:15,579 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-20 23:53:15,597 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1386121508:84 , srv1469513929:84 , srv1508566124:84 , srv1659953076:84 , srv1663578533:84 , srv1706782585:84 , srv1729095737:84 , srv1739436325:84 , srv1748064660:84 , srv1797089030:84 , srv1845218551:84 , srv1876449048:84 , srv1974813145:84 , srv2022534933:84 , srv364332787:84 , srv3867128:84 , srv441712652:84 , srv50397192:84 , srv560411501:84 , srv820923850:84 , srv823556248:84 , srv838471484:84 , srv842064555:84 , srv869229229:84 , srv879271378:84 , srv885452834:84 , srv911019524:84 , srv914470037:84 , srv981672554:84 , srv858667783:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-20 23:53:15,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:53:15,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv879271378=25, srv820923850=19, srv823556248=20, srv441712652=16, srv1659953076=3, srv1876449048=11, srv1663578533=4, srv1845218551=10, srv1748064660=8, srv914470037=28, srv560411501=18, srv842064555=22, srv1797089030=9, srv2022534933=13, srv50397192=17, srv838471484=21, srv869229229=24, srv1739436325=7, srv885452834=26, srv364332787=14, srv981672554=29, srv1706782585=5, srv1469513929=1, srv1729095737=6, srv858667783=23, srv1508566124=2, srv3867128=15, srv911019524=27, srv1386121508=0, srv1974813145=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:53:15,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:53:15,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-20 23:53:15,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1218661412070599, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:53:15,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912577039495928, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1218661412070599, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-20 23:55:15,626 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5486531 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912577039495928 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:55:15,634 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1386121508:90 , srv1469513929:90 , srv1508566124:90 , srv1659953076:90 , srv1663578533:90 , srv1706782585:90 , srv1729095737:90 , srv1739436325:90 , srv1748064660:90 , srv1797089030:90 , srv1845218551:90 , srv1876449048:90 , srv1974813145:90 , srv2022534933:90 , srv364332787:90 , srv3867128:90 , srv441712652:90 , srv50397192:90 , srv560411501:90 , srv820923850:90 , srv823556248:90 , srv838471484:90 , srv842064555:90 , srv858667783:90 , srv869229229:90 , srv879271378:90 , srv885452834:90 , srv911019524:90 , srv914470037:90 , srv981672554:90 }
2023-07-20 23:55:15,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-20 23:55:15,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv879271378=25, srv820923850=19, srv823556248=20, srv441712652=16, srv1659953076=3, srv1876449048=11, srv1663578533=4, srv1845218551=10, srv1748064660=8, srv914470037=28, srv560411501=18, srv842064555=22, srv1797089030=9, srv2022534933=13, srv50397192=17, srv838471484=21, srv869229229=24, srv1739436325=7, srv885452834=26, srv364332787=14, srv981672554=29, srv1706782585=5, srv1469513929=1, srv1729095737=6, srv858667783=23, srv1508566124=2, srv3867128=15, srv911019524=27, srv1386121508=0, srv1974813145=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-20 23:55:15,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-20 23:55:15,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-20 23:55:15,655 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-20 23:55:15,656 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1386121508:90 , srv1469513929:90 , srv1508566124:90 , srv1659953076:90 , srv1663578533:90 , srv1706782585:90 , srv1729095737:90 , srv1739436325:90 , srv1748064660:90 , srv1797089030:90 , srv1845218551:90 , srv1876449048:90 , srv1974813145:90 , srv2022534933:90 , srv364332787:90 , srv3867128:90 , srv441712652:90 , srv50397192:90 , srv560411501:90 , srv820923850:90 , srv823556248:90 , srv838471484:90 , srv842064555:90 , srv858667783:90 , srv869229229:90 , srv879271378:90 , srv885452834:90 , srv911019524:90 , srv914470037:90 , srv981672554:90 }
2023-07-20 23:55:15,671 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=197 (was 265), ProcessCount=171 (was 175), AvailableMemoryMB=6679 (was 6439) - AvailableMemoryMB LEAK? -