2023-12-05 17:58:33,666 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-05 17:58:33,981 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-05 17:58:34,094 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-05 17:58:34,158 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-05 17:58:34,165 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-05 17:58:34,181 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=221, ProcessCount=169, AvailableMemoryMB=6697
2023-12-05 17:58:34,184 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-05 17:58:34,184 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-05 17:58:34,316 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003601054:84 , srv10093919:84 , srv1037691583:84 , srv1038108873:84 , srv1044199401:84 , srv1070946093:84 , srv1087504237:84 , srv1173237043:84 , srv1203010972:84 , srv1268697571:84 , srv1291417340:84 , srv1349923879:84 , srv1363019627:84 , srv1379718932:84 , srv1412361459:84 , srv1417624511:84 , srv1422011359:84 , srv1425629218:84 , srv1482994354:84 , srv1487086778:84 , srv1489100295:84 , srv1494450593:84 , srv1507040386:84 , srv1519367654:84 , srv1547764802:84 , srv1561658819:84 , srv1579772468:84 , srv1600960048:84 , srv1602596938:84 , srv165655179:84 , srv1676759916:84 , srv1727751051:84 , srv1737516520:84 , srv1750334133:84 , srv1755331467:84 , srv1765082369:84 , srv1790100769:84 , srv1799216519:84 , srv1803643878:84 , srv1813582310:84 , srv1818946147:84 , srv1873289249:84 , srv187750970:84 , srv1911598425:84 , srv1919055807:84 , srv1978116060:84 , srv2020592196:84 , srv2085153029:84 , srv217329893:84 , srv261988288:84 , srv287819076:84 , srv294068111:84 , srv295030853:84 , srv337937709:84 , srv342042474:84 , srv348489768:84 , srv351168671:84 , srv357019033:84 , srv357951370:84 , srv383892236:84 , srv420104461:84 , srv473939651:84 , srv478301044:84 , srv487012483:84 , srv493017851:84 , srv493225097:84 , srv513199115:84 , srv52399700:84 , srv546488070:84 , srv583367934:84 , srv585641278:84 , srv58646776:84 , srv616201387:84 , srv629785399:84 , srv670238000:84 , srv673041368:84 , srv691872022:84 , srv711078217:84 , srv730957344:84 , srv739131820:84 , srv773370451:84 , srv78496174:84 , srv795620214:84 , srv802483129:84 , srv806658858:84 , srv818809456:84 , srv84473157:84 , srv858796341:84 , srv902946572:84 , srv917196338:84 , srv926741021:84 , srv945010984:84 , srv947624044:84 , srv952261699:84 , srv958870343:84 , srv963960954:84 , srv971199882:84 , srv988487705:84 , srv997688029:84 , srv1389110356:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-05 17:58:34,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:34,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1919055807=45, srv711078217=78, srv10093919=1, srv52399700=68, srv1173237043=7, srv342042474=55, srv2020592196=47, srv1790100769=37, srv295030853=53, srv1268697571=9, srv739131820=80, srv1579772468=27, srv478301044=63, srv2085153029=48, srv337937709=54, srv1737516520=33, srv351168671=57, srv1519367654=24, srv1547764802=25, srv902946572=89, srv1978116060=46, srv1507040386=23, srv1482994354=19, srv670238000=75, srv78496174=82, srv1818946147=41, srv963960954=96, srv261988288=50, srv1291417340=10, srv420104461=61, srv1070946093=5, srv217329893=49, srv1602596938=29, srv513199115=67, srv473939651=62, srv1389110356=14, srv795620214=83, srv493225097=66, srv691872022=77, srv1750334133=34, srv1561658819=26, srv294068111=52, srv58646776=72, srv673041368=76, srv997688029=99, srv493017851=65, srv585641278=71, srv926741021=91, srv187750970=43, srv287819076=51, srv1803643878=39, srv1349923879=11, srv773370451=81, srv546488070=69, srv1422011359=17, srv583367934=70, srv357019033=58, srv958870343=95, srv348489768=56, srv1487086778=20, srv357951370=59, srv84473157=87, srv1038108873=3, srv1676759916=31, srv730957344=79, srv1087504237=6, srv165655179=30, srv1813582310=40, srv818809456=86, srv616201387=73, srv1727751051=32, srv487012483=64, srv1765082369=36, srv1799216519=38, srv988487705=98, srv1203010972=8, srv1755331467=35, srv1412361459=15, srv1363019627=12, srv802483129=84, srv806658858=85, srv1037691583=2, srv1425629218=18, srv1911598425=44, srv1379718932=13, srv858796341=88, srv1600960048=28, srv1489100295=21, srv1003601054=0, srv1494450593=22, srv383892236=60, srv629785399=74, srv947624044=93, srv917196338=90, srv971199882=97, srv1873289249=42, srv1044199401=4, srv1417624511=16, srv952261699=94, srv945010984=92} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:34,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:34,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:34,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:34,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-05 17:58:34,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-05 17:58:34,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-05 17:58:34,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-05 17:58:34,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-05 17:58:34,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-05 17:58:34,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-05 17:58:34,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-05 17:58:34,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-05 17:58:34,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-05 17:58:34,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-05 17:58:34,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-05 17:58:34,584 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:34,585 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-05 17:58:34,585 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-05 17:58:34,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 475 ms to try 5000 different iterations.  Found a solution that moves 4715 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1803792279838197. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136919627330569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0587008623939312, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5238888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04136919627330569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24141666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:34,955 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1919055807:82 , srv902946572:82 , srv971199882:82 , srv1038108873:83 , srv1727751051:83 , srv1737516520:83 , srv1818946147:83 , srv187750970:83 , srv806658858:83 , srv1412361459:84 , srv1482994354:84 , srv1489100295:84 , srv261988288:84 , srv493225097:84 , srv583367934:84 , srv945010984:84 , srv947624044:84 , srv1203010972:85 , srv1422011359:85 , srv1425629218:85 , srv1561658819:85 , srv1600960048:85 , srv1676759916:85 , srv1765082369:85 , srv1873289249:85 , srv1911598425:85 , srv295030853:85 , srv342042474:85 , srv478301044:85 , srv673041368:85 , srv730957344:85 , srv773370451:85 , srv84473157:85 , srv926741021:85 , srv988487705:85 , srv1003601054:86 , srv10093919:86 , srv1044199401:86 , srv1268697571:86 , srv1349923879:86 , srv1494450593:86 , srv1547764802:86 , srv1579772468:86 , srv1750334133:86 , srv1755331467:86 , srv1790100769:86 , srv1799216519:86 , srv1813582310:86 , srv1978116060:86 , srv2085153029:86 , srv217329893:86 , srv287819076:86 , srv337937709:86 , srv351168671:86 , srv487012483:86 , srv52399700:86 , srv711078217:86 , srv818809456:86 , srv997688029:86 , srv1037691583:87 , srv1070946093:87 , srv1087504237:87 , srv1363019627:87 , srv1379718932:87 , srv357951370:87 , srv383892236:87 , srv473939651:87 , srv546488070:87 , srv585641278:87 , srv629785399:87 , srv691872022:87 , srv78496174:87 , srv795620214:87 , srv802483129:87 , srv952261699:87 , srv1487086778:88 , srv1507040386:88 , srv1602596938:88 , srv165655179:88 , srv1803643878:88 , srv2020592196:88 , srv348489768:88 , srv493017851:88 , srv58646776:88 , srv739131820:88 , srv958870343:88 , srv1417624511:89 , srv420104461:89 , srv513199115:89 , srv670238000:89 , srv858796341:89 , srv1173237043:90 , srv1519367654:90 , srv357019033:90 , srv963960954:90 , srv616201387:91 , srv1291417340:92 , srv294068111:92 , srv917196338:92 , srv1389110356:458 }
2023-12-05 17:58:34,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:34,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1919055807=45, srv711078217=78, srv10093919=1, srv52399700=68, srv1173237043=7, srv342042474=55, srv2020592196=47, srv1790100769=37, srv295030853=53, srv1268697571=9, srv739131820=80, srv1579772468=27, srv478301044=63, srv2085153029=48, srv337937709=54, srv1737516520=33, srv351168671=57, srv1519367654=24, srv1547764802=25, srv902946572=89, srv1978116060=46, srv1507040386=23, srv1482994354=19, srv670238000=75, srv78496174=82, srv1818946147=41, srv963960954=96, srv261988288=50, srv1291417340=10, srv420104461=61, srv1070946093=5, srv217329893=49, srv1602596938=29, srv513199115=67, srv473939651=62, srv1389110356=14, srv795620214=83, srv493225097=66, srv691872022=77, srv1750334133=34, srv1561658819=26, srv294068111=52, srv58646776=72, srv673041368=76, srv997688029=99, srv493017851=65, srv585641278=71, srv926741021=91, srv187750970=43, srv287819076=51, srv1803643878=39, srv1349923879=11, srv773370451=81, srv546488070=69, srv1422011359=17, srv583367934=70, srv357019033=58, srv958870343=95, srv348489768=56, srv1487086778=20, srv357951370=59, srv84473157=87, srv1038108873=3, srv1676759916=31, srv730957344=79, srv1087504237=6, srv165655179=30, srv1813582310=40, srv818809456=86, srv616201387=73, srv1727751051=32, srv487012483=64, srv1765082369=36, srv1799216519=38, srv988487705=98, srv1203010972=8, srv1755331467=35, srv1412361459=15, srv1363019627=12, srv802483129=84, srv806658858=85, srv1037691583=2, srv1425629218=18, srv1911598425=44, srv1379718932=13, srv858796341=88, srv1600960048=28, srv1489100295=21, srv1003601054=0, srv1494450593=22, srv383892236=60, srv629785399=74, srv947624044=93, srv917196338=90, srv971199882=97, srv1873289249=42, srv1044199401=4, srv1417624511=16, srv952261699=94, srv945010984=92} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:34,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:34,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:34,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:34,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:34,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-05 17:58:34,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-05 17:58:34,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-05 17:58:34,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-05 17:58:34,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-05 17:58:34,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-05 17:58:34,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-05 17:58:34,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-05 17:58:35,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136919627330569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0587008623939312, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04136919627330569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24141666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:35,113 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-05 17:58:35,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18034621284817307, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04136919627330569, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0587008623939312, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04136919627330569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17566666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24141666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-05 17:58:35,341 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 323 ms to try 5000 different iterations.  Found a solution that moves 2599 regions; Going from a computed imbalance of 0.18034621284817307 to a new imbalance of 0.01783134816327145. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023380583872248735); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03724520623121031, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2887777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023380583872248735); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:35,354 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv917196338:77 , srv294068111:81 , srv616201387:81 , srv858796341:81 , srv1173237043:82 , srv1507040386:82 , srv493017851:82 , srv513199115:82 , srv963960954:82 , srv1291417340:83 , srv1487086778:83 , srv1519367654:83 , srv1602596938:83 , srv1803643878:83 , srv357019033:83 , srv670238000:83 , srv739131820:83 , srv78496174:83 , srv958870343:83 , srv1070946093:84 , srv1087504237:84 , srv1379718932:84 , srv165655179:84 , srv1978116060:84 , srv2020592196:84 , srv348489768:84 , srv351168671:84 , srv357951370:84 , srv420104461:84 , srv473939651:84 , srv58646776:84 , srv691872022:84 , srv795620214:84 , srv10093919:85 , srv1363019627:85 , srv1417624511:85 , srv1547764802:85 , srv1561658819:85 , srv1790100769:85 , srv1799216519:85 , srv2085153029:85 , srv217329893:85 , srv585641278:85 , srv629785399:85 , srv773370451:85 , srv802483129:85 , srv1003601054:86 , srv1044199401:86 , srv1349923879:86 , srv1676759916:86 , srv1750334133:86 , srv1755331467:86 , srv1873289249:86 , srv287819076:86 , srv295030853:86 , srv337937709:86 , srv383892236:86 , srv546488070:86 , srv711078217:86 , srv818809456:86 , srv952261699:86 , srv988487705:86 , srv997688029:86 , srv1037691583:87 , srv1203010972:87 , srv1268697571:87 , srv1494450593:87 , srv1579772468:87 , srv1765082369:87 , srv1813582310:87 , srv1911598425:87 , srv261988288:87 , srv487012483:87 , srv493225097:87 , srv52399700:87 , srv673041368:87 , srv730957344:87 , srv926741021:87 , srv1425629218:88 , srv1818946147:88 , srv187750970:88 , srv478301044:88 , srv84473157:88 , srv902946572:88 , srv945010984:88 , srv947624044:88 , srv971199882:88 , srv1412361459:89 , srv1422011359:89 , srv1482994354:89 , srv1489100295:89 , srv1600960048:89 , srv342042474:89 , srv1038108873:90 , srv1727751051:90 , srv583367934:90 , srv806658858:90 , srv1737516520:91 , srv1919055807:91 , srv1389110356:524 }
2023-12-05 17:58:35,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:35,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1919055807=45, srv711078217=78, srv10093919=1, srv52399700=68, srv1173237043=7, srv342042474=55, srv2020592196=47, srv1790100769=37, srv295030853=53, srv1268697571=9, srv739131820=80, srv1579772468=27, srv478301044=63, srv2085153029=48, srv337937709=54, srv1737516520=33, srv351168671=57, srv1519367654=24, srv1547764802=25, srv902946572=89, srv1978116060=46, srv1507040386=23, srv1482994354=19, srv670238000=75, srv78496174=82, srv1818946147=41, srv963960954=96, srv261988288=50, srv1291417340=10, srv420104461=61, srv1070946093=5, srv217329893=49, srv1602596938=29, srv513199115=67, srv473939651=62, srv1389110356=14, srv795620214=83, srv493225097=66, srv691872022=77, srv1750334133=34, srv1561658819=26, srv294068111=52, srv58646776=72, srv673041368=76, srv997688029=99, srv493017851=65, srv585641278=71, srv926741021=91, srv187750970=43, srv287819076=51, srv1803643878=39, srv1349923879=11, srv773370451=81, srv546488070=69, srv1422011359=17, srv583367934=70, srv357019033=58, srv958870343=95, srv348489768=56, srv1487086778=20, srv357951370=59, srv84473157=87, srv1038108873=3, srv1676759916=31, srv730957344=79, srv1087504237=6, srv165655179=30, srv1813582310=40, srv818809456=86, srv616201387=73, srv1727751051=32, srv487012483=64, srv1765082369=36, srv1799216519=38, srv988487705=98, srv1203010972=8, srv1755331467=35, srv1412361459=15, srv1363019627=12, srv802483129=84, srv806658858=85, srv1037691583=2, srv1425629218=18, srv1911598425=44, srv1379718932=13, srv858796341=88, srv1600960048=28, srv1489100295=21, srv1003601054=0, srv1494450593=22, srv383892236=60, srv629785399=74, srv947624044=93, srv917196338=90, srv971199882=97, srv1873289249=42, srv1044199401=4, srv1417624511=16, srv952261699=94, srv945010984=92} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:35,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:35,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:35,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:35,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-05 17:58:35,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-05 17:58:35,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-05 17:58:35,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-05 17:58:35,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-05 17:58:35,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-05 17:58:35,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-05 17:58:35,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-05 17:58:35,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-05 17:58:35,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023380583872248735); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03724520623121031, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023380583872248735); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:35,517 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-05 17:58:35,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01781314957630525, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023380583872248735); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03724520623121031, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023380583872248735); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-05 17:58:35,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 282 ms to try 5000 different iterations.  Found a solution that moves 983 regions; Going from a computed imbalance of 0.01781314957630525 to a new imbalance of 6.903713077961024E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.10922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:35,711 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1037691583:83 , srv1422011359:83 , srv342042474:83 , srv1038108873:84 , srv1291417340:84 , srv1412361459:84 , srv1417624511:84 , srv1425629218:84 , srv1489100295:84 , srv1494450593:84 , srv1519367654:84 , srv1600960048:84 , srv1727751051:84 , srv1737516520:84 , srv383892236:84 , srv420104461:84 , srv478301044:84 , srv487012483:84 , srv546488070:84 , srv583367934:84 , srv84473157:84 , srv952261699:84 , srv1173237043:85 , srv1268697571:85 , srv1349923879:85 , srv1482994354:85 , srv1579772468:85 , srv165655179:85 , srv1750334133:85 , srv1755331467:85 , srv1813582310:85 , srv1919055807:85 , srv2020592196:85 , srv287819076:85 , srv294068111:85 , srv337937709:85 , srv348489768:85 , srv357019033:85 , srv52399700:85 , srv616201387:85 , srv711078217:85 , srv730957344:85 , srv806658858:85 , srv818809456:85 , srv945010984:85 , srv947624044:85 , srv997688029:85 , srv1003601054:86 , srv1044199401:86 , srv1087504237:86 , srv1203010972:86 , srv1363019627:86 , srv1547764802:86 , srv1602596938:86 , srv1765082369:86 , srv187750970:86 , srv1911598425:86 , srv2085153029:86 , srv217329893:86 , srv295030853:86 , srv493225097:86 , srv585641278:86 , srv629785399:86 , srv670238000:86 , srv673041368:86 , srv795620214:86 , srv802483129:86 , srv926741021:86 , srv958870343:86 , srv963960954:86 , srv1070946093:87 , srv1379718932:87 , srv1487086778:87 , srv1507040386:87 , srv1676759916:87 , srv1790100769:87 , srv1799216519:87 , srv1803643878:87 , srv1818946147:87 , srv1873289249:87 , srv1978116060:87 , srv261988288:87 , srv351168671:87 , srv357951370:87 , srv473939651:87 , srv493017851:87 , srv513199115:87 , srv58646776:87 , srv691872022:87 , srv739131820:87 , srv858796341:87 , srv988487705:87 , srv10093919:88 , srv1561658819:88 , srv773370451:88 , srv78496174:88 , srv971199882:88 , srv902946572:89 , srv917196338:89 , srv1389110356:520 }
2023-12-05 17:58:35,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:35,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1919055807=45, srv711078217=78, srv10093919=1, srv52399700=68, srv1173237043=7, srv342042474=55, srv2020592196=47, srv1790100769=37, srv295030853=53, srv1268697571=9, srv739131820=80, srv1579772468=27, srv478301044=63, srv2085153029=48, srv337937709=54, srv1737516520=33, srv351168671=57, srv1519367654=24, srv1547764802=25, srv902946572=89, srv1978116060=46, srv1507040386=23, srv1482994354=19, srv670238000=75, srv78496174=82, srv1818946147=41, srv963960954=96, srv261988288=50, srv1291417340=10, srv420104461=61, srv1070946093=5, srv217329893=49, srv1602596938=29, srv513199115=67, srv473939651=62, srv1389110356=14, srv795620214=83, srv493225097=66, srv691872022=77, srv1750334133=34, srv1561658819=26, srv294068111=52, srv58646776=72, srv673041368=76, srv997688029=99, srv493017851=65, srv585641278=71, srv926741021=91, srv187750970=43, srv287819076=51, srv1803643878=39, srv1349923879=11, srv773370451=81, srv546488070=69, srv1422011359=17, srv583367934=70, srv357019033=58, srv958870343=95, srv348489768=56, srv1487086778=20, srv357951370=59, srv84473157=87, srv1038108873=3, srv1676759916=31, srv730957344=79, srv1087504237=6, srv165655179=30, srv1813582310=40, srv818809456=86, srv616201387=73, srv1727751051=32, srv487012483=64, srv1765082369=36, srv1799216519=38, srv988487705=98, srv1203010972=8, srv1755331467=35, srv1412361459=15, srv1363019627=12, srv802483129=84, srv806658858=85, srv1037691583=2, srv1425629218=18, srv1911598425=44, srv1379718932=13, srv858796341=88, srv1600960048=28, srv1489100295=21, srv1003601054=0, srv1494450593=22, srv383892236=60, srv629785399=74, srv947624044=93, srv917196338=90, srv971199882=97, srv1873289249=42, srv1044199401=4, srv1417624511=16, srv952261699=94, srv945010984=92} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:35,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:35,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-05 17:58:35,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-05 17:58:35,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-05 17:58:35,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-05 17:58:35,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-05 17:58:35,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-05 17:58:35,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:35,808 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-05 17:58:35,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.834881946803759E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004979068295114799); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004979068295114799); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-05 17:58:36,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 320 ms to try 5000 different iterations.  Found a solution that moves 272 regions; Going from a computed imbalance of 6.834881946803759E-4 to a new imbalance of 2.200619618422514E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.030222222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:36,058 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1291417340:82 , srv10093919:83 , srv1044199401:83 , srv1873289249:83 , srv261988288:83 , srv58646776:83 , srv629785399:83 , srv902946572:83 , srv1003601054:84 , srv1070946093:84 , srv1268697571:84 , srv1363019627:84 , srv1379718932:84 , srv1482994354:84 , srv1487086778:84 , srv1561658819:84 , srv1600960048:84 , srv165655179:84 , srv1737516520:84 , srv1765082369:84 , srv1790100769:84 , srv1799216519:84 , srv1803643878:84 , srv1818946147:84 , srv1911598425:84 , srv1919055807:84 , srv287819076:84 , srv294068111:84 , srv357019033:84 , srv473939651:84 , srv493225097:84 , srv513199115:84 , srv52399700:84 , srv583367934:84 , srv585641278:84 , srv670238000:84 , srv673041368:84 , srv691872022:84 , srv739131820:84 , srv78496174:84 , srv802483129:84 , srv806658858:84 , srv917196338:84 , srv926741021:84 , srv963960954:84 , srv971199882:84 , srv988487705:84 , srv1037691583:85 , srv1038108873:85 , srv1087504237:85 , srv1173237043:85 , srv1203010972:85 , srv1349923879:85 , srv1412361459:85 , srv1417624511:85 , srv1422011359:85 , srv1425629218:85 , srv1489100295:85 , srv1494450593:85 , srv1507040386:85 , srv1519367654:85 , srv1547764802:85 , srv1579772468:85 , srv1602596938:85 , srv1676759916:85 , srv1727751051:85 , srv1750334133:85 , srv1755331467:85 , srv1813582310:85 , srv187750970:85 , srv1978116060:85 , srv2020592196:85 , srv2085153029:85 , srv217329893:85 , srv295030853:85 , srv337937709:85 , srv342042474:85 , srv348489768:85 , srv351168671:85 , srv357951370:85 , srv383892236:85 , srv420104461:85 , srv478301044:85 , srv487012483:85 , srv493017851:85 , srv546488070:85 , srv616201387:85 , srv711078217:85 , srv730957344:85 , srv773370451:85 , srv795620214:85 , srv818809456:85 , srv858796341:85 , srv945010984:85 , srv947624044:85 , srv952261699:85 , srv958870343:85 , srv997688029:85 , srv84473157:86 , srv1389110356:640 }
2023-12-05 17:58:36,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:36,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1919055807=45, srv711078217=78, srv10093919=1, srv52399700=68, srv1173237043=7, srv342042474=55, srv2020592196=47, srv1790100769=37, srv295030853=53, srv1268697571=9, srv739131820=80, srv1579772468=27, srv478301044=63, srv2085153029=48, srv337937709=54, srv1737516520=33, srv351168671=57, srv1519367654=24, srv1547764802=25, srv902946572=89, srv1978116060=46, srv1507040386=23, srv1482994354=19, srv670238000=75, srv78496174=82, srv1818946147=41, srv963960954=96, srv261988288=50, srv1291417340=10, srv420104461=61, srv1070946093=5, srv217329893=49, srv1602596938=29, srv513199115=67, srv473939651=62, srv1389110356=14, srv795620214=83, srv493225097=66, srv691872022=77, srv1750334133=34, srv1561658819=26, srv294068111=52, srv58646776=72, srv673041368=76, srv997688029=99, srv493017851=65, srv585641278=71, srv926741021=91, srv187750970=43, srv287819076=51, srv1803643878=39, srv1349923879=11, srv773370451=81, srv546488070=69, srv1422011359=17, srv583367934=70, srv357019033=58, srv958870343=95, srv348489768=56, srv1487086778=20, srv357951370=59, srv84473157=87, srv1038108873=3, srv1676759916=31, srv730957344=79, srv1087504237=6, srv165655179=30, srv1813582310=40, srv818809456=86, srv616201387=73, srv1727751051=32, srv487012483=64, srv1765082369=36, srv1799216519=38, srv988487705=98, srv1203010972=8, srv1755331467=35, srv1412361459=15, srv1363019627=12, srv802483129=84, srv806658858=85, srv1037691583=2, srv1425629218=18, srv1911598425=44, srv1379718932=13, srv858796341=88, srv1600960048=28, srv1489100295=21, srv1003601054=0, srv1494450593=22, srv383892236=60, srv629785399=74, srv947624044=93, srv917196338=90, srv971199882=97, srv1873289249=42, srv1044199401=4, srv1417624511=16, srv952261699=94, srv945010984=92} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:36,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:36,074 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:36,075 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-05 17:58:36,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-05 17:58:36,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-05 17:58:36,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-05 17:58:36,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-05 17:58:36,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-05 17:58:36,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-05 17:58:36,158 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=2.0101611476719935E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023210354127426377); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:36,158 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1291417340:82 , srv10093919:83 , srv1044199401:83 , srv1873289249:83 , srv261988288:83 , srv58646776:83 , srv629785399:83 , srv902946572:83 , srv1003601054:84 , srv1070946093:84 , srv1268697571:84 , srv1363019627:84 , srv1379718932:84 , srv1482994354:84 , srv1487086778:84 , srv1561658819:84 , srv1600960048:84 , srv165655179:84 , srv1737516520:84 , srv1765082369:84 , srv1790100769:84 , srv1799216519:84 , srv1803643878:84 , srv1818946147:84 , srv1911598425:84 , srv1919055807:84 , srv287819076:84 , srv294068111:84 , srv357019033:84 , srv473939651:84 , srv493225097:84 , srv513199115:84 , srv52399700:84 , srv583367934:84 , srv585641278:84 , srv670238000:84 , srv673041368:84 , srv691872022:84 , srv739131820:84 , srv78496174:84 , srv802483129:84 , srv806658858:84 , srv917196338:84 , srv926741021:84 , srv963960954:84 , srv971199882:84 , srv988487705:84 , srv1037691583:85 , srv1038108873:85 , srv1087504237:85 , srv1173237043:85 , srv1203010972:85 , srv1349923879:85 , srv1412361459:85 , srv1417624511:85 , srv1422011359:85 , srv1425629218:85 , srv1489100295:85 , srv1494450593:85 , srv1507040386:85 , srv1519367654:85 , srv1547764802:85 , srv1579772468:85 , srv1602596938:85 , srv1676759916:85 , srv1727751051:85 , srv1750334133:85 , srv1755331467:85 , srv1813582310:85 , srv187750970:85 , srv1978116060:85 , srv2020592196:85 , srv2085153029:85 , srv217329893:85 , srv295030853:85 , srv337937709:85 , srv342042474:85 , srv348489768:85 , srv351168671:85 , srv357951370:85 , srv383892236:85 , srv420104461:85 , srv478301044:85 , srv487012483:85 , srv493017851:85 , srv546488070:85 , srv616201387:85 , srv711078217:85 , srv730957344:85 , srv773370451:85 , srv795620214:85 , srv818809456:85 , srv858796341:85 , srv945010984:85 , srv947624044:85 , srv952261699:85 , srv958870343:85 , srv997688029:85 , srv84473157:86 , srv1389110356:640 }
2023-12-05 17:58:36,182 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=221 (was 221), ProcessCount=169 (was 169), AvailableMemoryMB=6233 (was 6697)
2023-12-05 17:58:36,190 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=221, ProcessCount=169, AvailableMemoryMB=6233
2023-12-05 17:58:36,191 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-05 17:58:36,191 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-05 17:58:36,200 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1049382035:84 , srv1077224500:84 , srv1090458135:84 , srv1288050269:84 , srv1310189459:84 , srv1319660272:84 , srv1427773486:84 , srv1434664621:84 , srv1577942094:84 , srv1599200469:84 , srv1604567973:84 , srv1709374742:84 , srv1925783033:84 , srv1964708270:84 , srv1986735149:84 , srv2134767486:84 , srv2138902874:84 , srv290961992:84 , srv394161724:84 , srv403664896:84 , srv428006196:84 , srv482214059:84 , srv572980180:84 , srv684227235:84 , srv801877081:84 , srv81214193:84 , srv892467481:84 , srv905422266:84 , srv930468849:84 , srv1832358853:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-05 17:58:36,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 17:58:36,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1964708270=14, srv801877081=25, srv1577942094=8, srv482214059=22, srv1709374742=11, srv1599200469=9, srv1434664621=7, srv930468849=29, srv905422266=28, srv428006196=21, srv1049382035=0, srv1925783033=13, srv2138902874=17, srv1986735149=15, srv1427773486=6, srv1090458135=2, srv1604567973=10, srv572980180=23, srv81214193=26, srv290961992=18, srv2134767486=16, srv394161724=19, srv892467481=27, srv403664896=20, srv1288050269=3, srv1832358853=12, srv684227235=24, srv1310189459=4, srv1319660272=5, srv1077224500=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 17:58:36,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 17:58:36,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-05 17:58:36,225 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1457632781784952, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 17:58:36,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912652338594211, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1457632781784952, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-05 18:00:36,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9628987 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912652338594211 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 18:00:36,220 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1049382035:90 , srv1077224500:90 , srv1090458135:90 , srv1288050269:90 , srv1310189459:90 , srv1319660272:90 , srv1427773486:90 , srv1434664621:90 , srv1577942094:90 , srv1599200469:90 , srv1604567973:90 , srv1709374742:90 , srv1832358853:90 , srv1925783033:90 , srv1964708270:90 , srv1986735149:90 , srv2134767486:90 , srv2138902874:90 , srv290961992:90 , srv394161724:90 , srv403664896:90 , srv428006196:90 , srv482214059:90 , srv572980180:90 , srv684227235:90 , srv801877081:90 , srv81214193:90 , srv892467481:90 , srv905422266:90 , srv930468849:90 }
2023-12-05 18:00:36,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-05 18:00:36,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1964708270=14, srv801877081=25, srv1577942094=8, srv482214059=22, srv1709374742=11, srv1599200469=9, srv1434664621=7, srv930468849=29, srv905422266=28, srv428006196=21, srv1049382035=0, srv1925783033=13, srv2138902874=17, srv1986735149=15, srv1427773486=6, srv1090458135=2, srv1604567973=10, srv572980180=23, srv81214193=26, srv290961992=18, srv2134767486=16, srv394161724=19, srv892467481=27, srv403664896=20, srv1288050269=3, srv1832358853=12, srv684227235=24, srv1310189459=4, srv1319660272=5, srv1077224500=1} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-05 18:00:36,224 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-05 18:00:36,225 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-05 18:00:36,234 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-05 18:00:36,234 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1049382035:90 , srv1077224500:90 , srv1090458135:90 , srv1288050269:90 , srv1310189459:90 , srv1319660272:90 , srv1427773486:90 , srv1434664621:90 , srv1577942094:90 , srv1599200469:90 , srv1604567973:90 , srv1709374742:90 , srv1832358853:90 , srv1925783033:90 , srv1964708270:90 , srv1986735149:90 , srv2134767486:90 , srv2138902874:90 , srv290961992:90 , srv394161724:90 , srv403664896:90 , srv428006196:90 , srv482214059:90 , srv572980180:90 , srv684227235:90 , srv801877081:90 , srv81214193:90 , srv892467481:90 , srv905422266:90 , srv930468849:90 }
2023-12-05 18:00:36,246 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=130 (was 221), ProcessCount=167 (was 169), AvailableMemoryMB=7535 (was 6233) - AvailableMemoryMB LEAK? -