2023-05-30 23:53:25,786 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-30 23:53:26,052 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-30 23:53:26,149 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-30 23:53:26,204 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-30 23:53:26,211 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-30 23:53:26,224 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=298, ProcessCount=170, AvailableMemoryMB=3820
2023-05-30 23:53:26,225 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-30 23:53:26,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-30 23:53:26,350 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015684309:84 , srv1017843144:84 , srv1053552390:84 , srv1061395009:84 , srv1081048004:84 , srv1089206879:84 , srv1123351206:84 , srv114047314:84 , srv1154739965:84 , srv1232616727:84 , srv1246852492:84 , srv1256101517:84 , srv1281130936:84 , srv1290009534:84 , srv1318048087:84 , srv1352293177:84 , srv1355136447:84 , srv1358383680:84 , srv1394520382:84 , srv142514822:84 , srv1444874867:84 , srv1482404949:84 , srv1525212028:84 , srv1587130478:84 , srv1588412865:84 , srv1597346246:84 , srv1601289191:84 , srv1615482749:84 , srv1639925387:84 , srv1705139245:84 , srv1722377617:84 , srv1742658893:84 , srv1745613734:84 , srv1801896243:84 , srv1802155514:84 , srv1822331339:84 , srv1838766825:84 , srv1840595734:84 , srv1866991135:84 , srv194525356:84 , srv1955621186:84 , srv1972274235:84 , srv197872042:84 , srv1992858418:84 , srv200772074:84 , srv2022600214:84 , srv2034434682:84 , srv2059127596:84 , srv2080617441:84 , srv2083343683:84 , srv239217638:84 , srv249253306:84 , srv257491697:84 , srv259251947:84 , srv259628378:84 , srv272419673:84 , srv321253797:84 , srv355202851:84 , srv368889650:84 , srv384031419:84 , srv392567377:84 , srv434665521:84 , srv487975160:84 , srv490297738:84 , srv491405280:84 , srv491794167:84 , srv496553548:84 , srv514135881:84 , srv518056151:84 , srv531222828:84 , srv536229125:84 , srv577415363:84 , srv578530359:84 , srv596080867:84 , srv626174563:84 , srv628315435:84 , srv642982192:84 , srv643579830:84 , srv652698489:84 , srv663487198:84 , srv669913712:84 , srv675123427:84 , srv688075776:84 , srv702123254:84 , srv712382032:84 , srv725431904:84 , srv773611698:84 , srv802356127:84 , srv807934983:84 , srv814340706:84 , srv824865156:84 , srv83256620:84 , srv847354789:84 , srv855163984:84 , srv86736773:84 , srv908996497:84 , srv932292948:84 , srv959015694:84 , srv988899700:84 , srv821465543:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-30 23:53:26,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:26,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1061395009=3, srv1615482749=27, srv83256620=92, srv1992858418=43, srv2083343683=49, srv496553548=66, srv1017843144=1, srv932292948=97, srv1482404949=21, srv807934983=88, srv491405280=64, srv1053552390=2, srv194525356=39, srv1444874867=20, srv1588412865=24, srv1246852492=10, srv1089206879=5, srv1801896243=33, srv855163984=94, srv259628378=54, srv1597346246=25, srv1955621186=40, srv355202851=57, srv518056151=68, srv2080617441=48, srv663487198=79, srv688075776=82, srv1866991135=38, srv725431904=85, srv249253306=51, srv1355136447=16, srv596080867=73, srv86736773=95, srv259251947=53, srv847354789=93, srv490297738=63, srv642982192=76, srv1123351206=6, srv1822331339=35, srv1318048087=14, srv114047314=7, srv1015684309=0, srv702123254=83, srv773611698=86, srv239217638=50, srv434665521=61, srv1745613734=32, srv988899700=99, srv1290009534=13, srv1972274235=41, srv1838766825=36, srv368889650=58, srv1256101517=11, srv2022600214=45, srv908996497=96, srv1722377617=30, srv578530359=72, srv1281130936=12, srv1525212028=22, srv814340706=89, srv1358383680=17, srv669913712=80, srv531222828=69, srv1352293177=15, srv821465543=90, srv959015694=98, srv1705139245=29, srv536229125=70, srv628315435=75, srv643579830=77, srv491794167=65, srv197872042=42, srv257491697=52, srv200772074=44, srv1742658893=31, srv626174563=74, srv1601289191=26, srv2034434682=46, srv1154739965=8, srv824865156=91, srv1394520382=18, srv272419673=55, srv487975160=62, srv384031419=59, srv652698489=78, srv2059127596=47, srv577415363=71, srv321253797=56, srv1639925387=28, srv1232616727=9, srv675123427=81, srv392567377=60, srv142514822=19, srv514135881=67, srv712382032=84, srv1081048004=4, srv1587130478=23, srv802356127=87, srv1802155514=34, srv1840595734=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:26,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:26,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:26,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-30 23:53:26,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-30 23:53:26,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-30 23:53:26,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-30 23:53:26,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-30 23:53:26,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-30 23:53:26,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-30 23:53:26,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-30 23:53:26,593 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:26,593 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-30 23:53:26,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-30 23:53:26,899 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 444 ms to try 5000 different iterations.  Found a solution that moves 4655 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18851709165164304. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039377878103709664, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5172222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039377878103709664, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:26,912 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1089206879:82 , srv1802155514:82 , srv249253306:82 , srv988899700:82 , srv1061395009:83 , srv1597346246:83 , srv1742658893:83 , srv200772074:83 , srv490297738:83 , srv642982192:83 , srv675123427:83 , srv1318048087:84 , srv142514822:84 , srv1601289191:84 , srv1639925387:84 , srv197872042:84 , srv1992858418:84 , srv2080617441:84 , srv514135881:84 , srv518056151:84 , srv663487198:84 , srv1017843144:85 , srv1256101517:85 , srv1352293177:85 , srv1482404949:85 , srv1822331339:85 , srv1838766825:85 , srv2022600214:85 , srv2083343683:85 , srv259251947:85 , srv578530359:85 , srv626174563:85 , srv628315435:85 , srv652698489:85 , srv669913712:85 , srv688075776:85 , srv86736773:85 , srv908996497:85 , srv1053552390:86 , srv1123351206:86 , srv1154739965:86 , srv1281130936:86 , srv1358383680:86 , srv1394520382:86 , srv1588412865:86 , srv1705139245:86 , srv194525356:86 , srv1955621186:86 , srv2034434682:86 , srv2059127596:86 , srv368889650:86 , srv536229125:86 , srv1015684309:87 , srv114047314:87 , srv1232616727:87 , srv1587130478:87 , srv1801896243:87 , srv1866991135:87 , srv496553548:87 , srv702123254:87 , srv725431904:87 , srv773611698:87 , srv814340706:87 , srv83256620:87 , srv932292948:87 , srv959015694:87 , srv1081048004:88 , srv1290009534:88 , srv1355136447:88 , srv1722377617:88 , srv1745613734:88 , srv239217638:88 , srv257491697:88 , srv321253797:88 , srv392567377:88 , srv434665521:88 , srv491405280:88 , srv577415363:88 , srv596080867:88 , srv643579830:88 , srv712382032:88 , srv1444874867:89 , srv272419673:89 , srv355202851:89 , srv487975160:89 , srv531222828:89 , srv802356127:89 , srv824865156:89 , srv847354789:89 , srv855163984:89 , srv1246852492:90 , srv1525212028:90 , srv1840595734:90 , srv1615482749:91 , srv384031419:91 , srv491794167:91 , srv1972274235:92 , srv259628378:93 , srv807934983:95 , srv821465543:440 }
2023-05-30 23:53:26,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:26,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1061395009=3, srv1615482749=27, srv83256620=92, srv1992858418=43, srv2083343683=49, srv496553548=66, srv1017843144=1, srv932292948=97, srv1482404949=21, srv807934983=88, srv491405280=64, srv1053552390=2, srv194525356=39, srv1444874867=20, srv1588412865=24, srv1246852492=10, srv1089206879=5, srv1801896243=33, srv855163984=94, srv259628378=54, srv1597346246=25, srv1955621186=40, srv355202851=57, srv518056151=68, srv2080617441=48, srv663487198=79, srv688075776=82, srv1866991135=38, srv725431904=85, srv249253306=51, srv1355136447=16, srv596080867=73, srv86736773=95, srv259251947=53, srv847354789=93, srv490297738=63, srv642982192=76, srv1123351206=6, srv1822331339=35, srv1318048087=14, srv114047314=7, srv1015684309=0, srv702123254=83, srv773611698=86, srv239217638=50, srv434665521=61, srv1745613734=32, srv988899700=99, srv1290009534=13, srv1972274235=41, srv1838766825=36, srv368889650=58, srv1256101517=11, srv2022600214=45, srv908996497=96, srv1722377617=30, srv578530359=72, srv1281130936=12, srv1525212028=22, srv814340706=89, srv1358383680=17, srv669913712=80, srv531222828=69, srv1352293177=15, srv821465543=90, srv959015694=98, srv1705139245=29, srv536229125=70, srv628315435=75, srv643579830=77, srv491794167=65, srv197872042=42, srv257491697=52, srv200772074=44, srv1742658893=31, srv626174563=74, srv1601289191=26, srv2034434682=46, srv1154739965=8, srv824865156=91, srv1394520382=18, srv272419673=55, srv487975160=62, srv384031419=59, srv652698489=78, srv2059127596=47, srv577415363=71, srv321253797=56, srv1639925387=28, srv1232616727=9, srv675123427=81, srv392567377=60, srv142514822=19, srv514135881=67, srv712382032=84, srv1081048004=4, srv1587130478=23, srv802356127=87, srv1802155514=34, srv1840595734=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:26,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:26,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:26,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:26,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:26,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:26,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:26,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-30 23:53:26,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-30 23:53:26,942 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-30 23:53:26,943 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-30 23:53:26,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-30 23:53:26,945 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-30 23:53:26,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-30 23:53:26,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-30 23:53:26,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-30 23:53:26,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-30 23:53:27,092 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039377878103709664, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039377878103709664, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:27,093 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-30 23:53:27,093 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.188484496644976, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039377878103709664, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053783871430366184, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039377878103709664, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-30 23:53:27,497 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 522 ms to try 5000 different iterations.  Found a solution that moves 2688 regions; Going from a computed imbalance of 0.188484496644976 to a new imbalance of 0.01466894161780276. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02002990655876138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03148405614347985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2986666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02002990655876138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:27,510 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv807934983:78 , srv259628378:80 , srv1015684309:82 , srv1246852492:82 , srv1355136447:82 , srv1615482749:82 , srv1972274235:82 , srv257491697:82 , srv491794167:82 , srv824865156:82 , srv855163984:82 , srv1081048004:83 , srv1525212028:83 , srv1722377617:83 , srv1840595734:83 , srv487975160:83 , srv802356127:83 , srv959015694:83 , srv272419673:84 , srv321253797:84 , srv355202851:84 , srv384031419:84 , srv392567377:84 , srv491405280:84 , srv596080867:84 , srv702123254:84 , srv847354789:84 , srv114047314:85 , srv1232616727:85 , srv1394520382:85 , srv1444874867:85 , srv1588412865:85 , srv1705139245:85 , srv1838766825:85 , srv2034434682:85 , srv239217638:85 , srv434665521:85 , srv496553548:85 , srv536229125:85 , srv577415363:85 , srv578530359:85 , srv652698489:85 , srv712382032:85 , srv814340706:85 , srv1123351206:86 , srv1281130936:86 , srv1290009534:86 , srv1358383680:86 , srv1482404949:86 , srv1587130478:86 , srv1745613734:86 , srv1801896243:86 , srv1866991135:86 , srv194525356:86 , srv1955621186:86 , srv2022600214:86 , srv2059127596:86 , srv531222828:86 , srv628315435:86 , srv643579830:86 , srv773611698:86 , srv908996497:86 , srv932292948:86 , srv1017843144:87 , srv1053552390:87 , srv1318048087:87 , srv1992858418:87 , srv2080617441:87 , srv368889650:87 , srv490297738:87 , srv514135881:87 , srv518056151:87 , srv626174563:87 , srv663487198:87 , srv725431904:87 , srv1061395009:88 , srv1256101517:88 , srv1352293177:88 , srv142514822:88 , srv1601289191:88 , srv197872042:88 , srv200772074:88 , srv259251947:88 , srv688075776:88 , srv83256620:88 , srv86736773:88 , srv988899700:88 , srv1154739965:89 , srv1597346246:89 , srv1639925387:89 , srv1742658893:89 , srv1822331339:89 , srv2083343683:89 , srv1089206879:90 , srv1802155514:90 , srv642982192:90 , srv669913712:90 , srv675123427:90 , srv249253306:92 , srv821465543:512 }
2023-05-30 23:53:27,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:27,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1061395009=3, srv1615482749=27, srv83256620=92, srv1992858418=43, srv2083343683=49, srv496553548=66, srv1017843144=1, srv932292948=97, srv1482404949=21, srv807934983=88, srv491405280=64, srv1053552390=2, srv194525356=39, srv1444874867=20, srv1588412865=24, srv1246852492=10, srv1089206879=5, srv1801896243=33, srv855163984=94, srv259628378=54, srv1597346246=25, srv1955621186=40, srv355202851=57, srv518056151=68, srv2080617441=48, srv663487198=79, srv688075776=82, srv1866991135=38, srv725431904=85, srv249253306=51, srv1355136447=16, srv596080867=73, srv86736773=95, srv259251947=53, srv847354789=93, srv490297738=63, srv642982192=76, srv1123351206=6, srv1822331339=35, srv1318048087=14, srv114047314=7, srv1015684309=0, srv702123254=83, srv773611698=86, srv239217638=50, srv434665521=61, srv1745613734=32, srv988899700=99, srv1290009534=13, srv1972274235=41, srv1838766825=36, srv368889650=58, srv1256101517=11, srv2022600214=45, srv908996497=96, srv1722377617=30, srv578530359=72, srv1281130936=12, srv1525212028=22, srv814340706=89, srv1358383680=17, srv669913712=80, srv531222828=69, srv1352293177=15, srv821465543=90, srv959015694=98, srv1705139245=29, srv536229125=70, srv628315435=75, srv643579830=77, srv491794167=65, srv197872042=42, srv257491697=52, srv200772074=44, srv1742658893=31, srv626174563=74, srv1601289191=26, srv2034434682=46, srv1154739965=8, srv824865156=91, srv1394520382=18, srv272419673=55, srv487975160=62, srv384031419=59, srv652698489=78, srv2059127596=47, srv577415363=71, srv321253797=56, srv1639925387=28, srv1232616727=9, srv675123427=81, srv392567377=60, srv142514822=19, srv514135881=67, srv712382032=84, srv1081048004=4, srv1587130478=23, srv802356127=87, srv1802155514=34, srv1840595734=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:27,536 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:27,537 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:27,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-30 23:53:27,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-30 23:53:27,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-30 23:53:27,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-30 23:53:27,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-30 23:53:27,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-30 23:53:27,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-30 23:53:27,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-30 23:53:27,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-30 23:53:27,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-30 23:53:27,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-30 23:53:27,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-30 23:53:27,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02002990655876138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03148405614347985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02002990655876138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:27,708 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-30 23:53:27,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014650119839516827, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02002990655876138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03148405614347985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02002990655876138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-30 23:53:28,107 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 546 ms to try 5000 different iterations.  Found a solution that moves 1023 regions; Going from a computed imbalance of 0.014650119839516827 to a new imbalance of 7.838843676892763E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00454981708595525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007683410204338025); MoveCostFunction : (multiplier=7.0, imbalance=0.11366666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00454981708595525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:28,109 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1154739965:83 , srv1587130478:83 , srv2083343683:83 , srv249253306:83 , srv384031419:83 , srv669913712:83 , srv725431904:83 , srv83256620:83 , srv1256101517:84 , srv1290009534:84 , srv1352293177:84 , srv1444874867:84 , srv1525212028:84 , srv1615482749:84 , srv1639925387:84 , srv1745613734:84 , srv1822331339:84 , srv1840595734:84 , srv1972274235:84 , srv239217638:84 , srv272419673:84 , srv368889650:84 , srv434665521:84 , srv531222828:84 , srv577415363:84 , srv642982192:84 , srv643579830:84 , srv712382032:84 , srv814340706:84 , srv1017843144:85 , srv1232616727:85 , srv1246852492:85 , srv1281130936:85 , srv142514822:85 , srv1742658893:85 , srv1801896243:85 , srv1802155514:85 , srv1866991135:85 , srv194525356:85 , srv1955621186:85 , srv2059127596:85 , srv259251947:85 , srv355202851:85 , srv491405280:85 , srv491794167:85 , srv496553548:85 , srv596080867:85 , srv626174563:85 , srv675123427:85 , srv688075776:85 , srv802356127:85 , srv847354789:85 , srv86736773:85 , srv932292948:85 , srv1053552390:86 , srv1081048004:86 , srv1089206879:86 , srv1123351206:86 , srv114047314:86 , srv1394520382:86 , srv1597346246:86 , srv1601289191:86 , srv197872042:86 , srv2022600214:86 , srv2034434682:86 , srv259628378:86 , srv321253797:86 , srv392567377:86 , srv487975160:86 , srv514135881:86 , srv518056151:86 , srv663487198:86 , srv773611698:86 , srv807934983:86 , srv908996497:86 , srv1061395009:87 , srv1318048087:87 , srv1355136447:87 , srv1358383680:87 , srv1482404949:87 , srv1588412865:87 , srv1705139245:87 , srv1722377617:87 , srv1992858418:87 , srv2080617441:87 , srv536229125:87 , srv578530359:87 , srv628315435:87 , srv652698489:87 , srv824865156:87 , srv855163984:87 , srv959015694:87 , srv988899700:87 , srv1838766825:88 , srv200772074:88 , srv257491697:88 , srv490297738:88 , srv702123254:88 , srv1015684309:89 , srv821465543:546 }
2023-05-30 23:53:28,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:28,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1061395009=3, srv1615482749=27, srv83256620=92, srv1992858418=43, srv2083343683=49, srv496553548=66, srv1017843144=1, srv932292948=97, srv1482404949=21, srv807934983=88, srv491405280=64, srv1053552390=2, srv194525356=39, srv1444874867=20, srv1588412865=24, srv1246852492=10, srv1089206879=5, srv1801896243=33, srv855163984=94, srv259628378=54, srv1597346246=25, srv1955621186=40, srv355202851=57, srv518056151=68, srv2080617441=48, srv663487198=79, srv688075776=82, srv1866991135=38, srv725431904=85, srv249253306=51, srv1355136447=16, srv596080867=73, srv86736773=95, srv259251947=53, srv847354789=93, srv490297738=63, srv642982192=76, srv1123351206=6, srv1822331339=35, srv1318048087=14, srv114047314=7, srv1015684309=0, srv702123254=83, srv773611698=86, srv239217638=50, srv434665521=61, srv1745613734=32, srv988899700=99, srv1290009534=13, srv1972274235=41, srv1838766825=36, srv368889650=58, srv1256101517=11, srv2022600214=45, srv908996497=96, srv1722377617=30, srv578530359=72, srv1281130936=12, srv1525212028=22, srv814340706=89, srv1358383680=17, srv669913712=80, srv531222828=69, srv1352293177=15, srv821465543=90, srv959015694=98, srv1705139245=29, srv536229125=70, srv628315435=75, srv643579830=77, srv491794167=65, srv197872042=42, srv257491697=52, srv200772074=44, srv1742658893=31, srv626174563=74, srv1601289191=26, srv2034434682=46, srv1154739965=8, srv824865156=91, srv1394520382=18, srv272419673=55, srv487975160=62, srv384031419=59, srv652698489=78, srv2059127596=47, srv577415363=71, srv321253797=56, srv1639925387=28, srv1232616727=9, srv675123427=81, srv392567377=60, srv142514822=19, srv514135881=67, srv712382032=84, srv1081048004=4, srv1587130478=23, srv802356127=87, srv1802155514=34, srv1840595734=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:28,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:28,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:28,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:28,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-30 23:53:28,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-30 23:53:28,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-30 23:53:28,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-30 23:53:28,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-30 23:53:28,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-30 23:53:28,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-30 23:53:28,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-30 23:53:28,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-30 23:53:28,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-30 23:53:28,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-30 23:53:28,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-30 23:53:28,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-30 23:53:28,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-30 23:53:28,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-30 23:53:28,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-30 23:53:28,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00454981708595525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007683410204338025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00454981708595525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:28,308 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-30 23:53:28,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.767211685871519E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00454981708595525); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007683410204338025); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00454981708595525); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-30 23:53:28,634 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 466 ms to try 5000 different iterations.  Found a solution that moves 239 regions; Going from a computed imbalance of 7.767211685871519E-4 to a new imbalance of 2.126690702209265E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.026555555555555554, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:28,635 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1053552390:83 , srv1358383680:83 , srv200772074:83 , srv259628378:83 , srv487975160:83 , srv531222828:83 , srv702123254:83 , srv773611698:83 , srv807934983:83 , srv1015684309:84 , srv1089206879:84 , srv1123351206:84 , srv114047314:84 , srv1154739965:84 , srv1290009534:84 , srv1318048087:84 , srv1444874867:84 , srv1482404949:84 , srv1588412865:84 , srv1597346246:84 , srv1601289191:84 , srv1705139245:84 , srv1722377617:84 , srv1801896243:84 , srv1838766825:84 , srv1866991135:84 , srv1972274235:84 , srv197872042:84 , srv1992858418:84 , srv2034434682:84 , srv2080617441:84 , srv239217638:84 , srv257491697:84 , srv259251947:84 , srv321253797:84 , srv355202851:84 , srv384031419:84 , srv392567377:84 , srv490297738:84 , srv491794167:84 , srv536229125:84 , srv626174563:84 , srv643579830:84 , srv669913712:84 , srv675123427:84 , srv688075776:84 , srv824865156:84 , srv847354789:84 , srv855163984:84 , srv86736773:84 , srv932292948:84 , srv988899700:84 , srv1017843144:85 , srv1061395009:85 , srv1081048004:85 , srv1232616727:85 , srv1246852492:85 , srv1256101517:85 , srv1281130936:85 , srv1352293177:85 , srv1355136447:85 , srv1394520382:85 , srv142514822:85 , srv1525212028:85 , srv1615482749:85 , srv1639925387:85 , srv1742658893:85 , srv1745613734:85 , srv1802155514:85 , srv1822331339:85 , srv1840595734:85 , srv194525356:85 , srv2022600214:85 , srv2059127596:85 , srv2083343683:85 , srv249253306:85 , srv272419673:85 , srv368889650:85 , srv434665521:85 , srv491405280:85 , srv496553548:85 , srv514135881:85 , srv518056151:85 , srv577415363:85 , srv578530359:85 , srv596080867:85 , srv628315435:85 , srv642982192:85 , srv652698489:85 , srv663487198:85 , srv712382032:85 , srv725431904:85 , srv802356127:85 , srv814340706:85 , srv83256620:85 , srv908996497:85 , srv959015694:85 , srv1587130478:86 , srv1955621186:86 , srv821465543:644 }
2023-05-30 23:53:28,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:28,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1061395009=3, srv1615482749=27, srv83256620=92, srv1992858418=43, srv2083343683=49, srv496553548=66, srv1017843144=1, srv932292948=97, srv1482404949=21, srv807934983=88, srv491405280=64, srv1053552390=2, srv194525356=39, srv1444874867=20, srv1588412865=24, srv1246852492=10, srv1089206879=5, srv1801896243=33, srv855163984=94, srv259628378=54, srv1597346246=25, srv1955621186=40, srv355202851=57, srv518056151=68, srv2080617441=48, srv663487198=79, srv688075776=82, srv1866991135=38, srv725431904=85, srv249253306=51, srv1355136447=16, srv596080867=73, srv86736773=95, srv259251947=53, srv847354789=93, srv490297738=63, srv642982192=76, srv1123351206=6, srv1822331339=35, srv1318048087=14, srv114047314=7, srv1015684309=0, srv702123254=83, srv773611698=86, srv239217638=50, srv434665521=61, srv1745613734=32, srv988899700=99, srv1290009534=13, srv1972274235=41, srv1838766825=36, srv368889650=58, srv1256101517=11, srv2022600214=45, srv908996497=96, srv1722377617=30, srv578530359=72, srv1281130936=12, srv1525212028=22, srv814340706=89, srv1358383680=17, srv669913712=80, srv531222828=69, srv1352293177=15, srv821465543=90, srv959015694=98, srv1705139245=29, srv536229125=70, srv628315435=75, srv643579830=77, srv491794167=65, srv197872042=42, srv257491697=52, srv200772074=44, srv1742658893=31, srv626174563=74, srv1601289191=26, srv2034434682=46, srv1154739965=8, srv824865156=91, srv1394520382=18, srv272419673=55, srv487975160=62, srv384031419=59, srv652698489=78, srv2059127596=47, srv577415363=71, srv321253797=56, srv1639925387=28, srv1232616727=9, srv675123427=81, srv392567377=60, srv142514822=19, srv514135881=67, srv712382032=84, srv1081048004=4, srv1587130478=23, srv802356127=87, srv1802155514=34, srv1840595734=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:28,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:28,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:28,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:28,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:28,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-30 23:53:28,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-30 23:53:28,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-30 23:53:28,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-30 23:53:28,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-30 23:53:28,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-30 23:53:28,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-30 23:53:28,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-30 23:53:28,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-30 23:53:28,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.959339325336565E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:28,796 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1053552390:83 , srv1358383680:83 , srv200772074:83 , srv259628378:83 , srv487975160:83 , srv531222828:83 , srv702123254:83 , srv773611698:83 , srv807934983:83 , srv1015684309:84 , srv1089206879:84 , srv1123351206:84 , srv114047314:84 , srv1154739965:84 , srv1290009534:84 , srv1318048087:84 , srv1444874867:84 , srv1482404949:84 , srv1588412865:84 , srv1597346246:84 , srv1601289191:84 , srv1705139245:84 , srv1722377617:84 , srv1801896243:84 , srv1838766825:84 , srv1866991135:84 , srv1972274235:84 , srv197872042:84 , srv1992858418:84 , srv2034434682:84 , srv2080617441:84 , srv239217638:84 , srv257491697:84 , srv259251947:84 , srv321253797:84 , srv355202851:84 , srv384031419:84 , srv392567377:84 , srv490297738:84 , srv491794167:84 , srv536229125:84 , srv626174563:84 , srv643579830:84 , srv669913712:84 , srv675123427:84 , srv688075776:84 , srv824865156:84 , srv847354789:84 , srv855163984:84 , srv86736773:84 , srv932292948:84 , srv988899700:84 , srv1017843144:85 , srv1061395009:85 , srv1081048004:85 , srv1232616727:85 , srv1246852492:85 , srv1256101517:85 , srv1281130936:85 , srv1352293177:85 , srv1355136447:85 , srv1394520382:85 , srv142514822:85 , srv1525212028:85 , srv1615482749:85 , srv1639925387:85 , srv1742658893:85 , srv1745613734:85 , srv1802155514:85 , srv1822331339:85 , srv1840595734:85 , srv194525356:85 , srv2022600214:85 , srv2059127596:85 , srv2083343683:85 , srv249253306:85 , srv272419673:85 , srv368889650:85 , srv434665521:85 , srv491405280:85 , srv496553548:85 , srv514135881:85 , srv518056151:85 , srv577415363:85 , srv578530359:85 , srv596080867:85 , srv628315435:85 , srv642982192:85 , srv652698489:85 , srv663487198:85 , srv712382032:85 , srv725431904:85 , srv802356127:85 , srv814340706:85 , srv83256620:85 , srv908996497:85 , srv959015694:85 , srv1587130478:86 , srv1955621186:86 , srv821465543:644 }
2023-05-30 23:53:28,823 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=298 (was 298), ProcessCount=170 (was 170), AvailableMemoryMB=3324 (was 3820)
2023-05-30 23:53:28,833 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=298, ProcessCount=170, AvailableMemoryMB=3324
2023-05-30 23:53:28,833 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-30 23:53:28,834 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-30 23:53:28,842 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1009056219:84 , srv1010001808:84 , srv1011120557:84 , srv1037042113:84 , srv112524859:84 , srv1180264461:84 , srv1311817421:84 , srv1443783857:84 , srv1483015092:84 , srv1610517590:84 , srv1669883417:84 , srv1857222085:84 , srv2007387437:84 , srv2033844707:84 , srv2057312481:84 , srv2080204621:84 , srv249886313:84 , srv332826375:84 , srv354081565:84 , srv459559218:84 , srv501411083:84 , srv53828647:84 , srv668843480:84 , srv675783815:84 , srv705530819:84 , srv806282589:84 , srv912789701:84 , srv950444858:84 , srv975452874:84 , srv1354127973:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-30 23:53:28,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:53:28,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2057312481=15, srv1010001808=1, srv332826375=18, srv806282589=26, srv501411083=21, srv950444858=28, srv1857222085=12, srv1180264461=5, srv912789701=27, srv2007387437=13, srv2033844707=14, srv1311817421=6, srv705530819=25, srv1011120557=2, srv459559218=20, srv1483015092=9, srv975452874=29, srv1009056219=0, srv249886313=17, srv2080204621=16, srv675783815=24, srv1037042113=3, srv1610517590=10, srv53828647=22, srv112524859=4, srv1443783857=8, srv1669883417=11, srv668843480=23, srv1354127973=7, srv354081565=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:53:28,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:53:28,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:53:28,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:53:28,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:53:28,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:53:28,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:53:28,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:53:28,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:53:28,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-30 23:53:28,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1434788725922946, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:53:28,866 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912645140506802, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1434788725922946, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-30 23:55:28,855 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5921832 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912645140506802 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:55:28,860 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1009056219:90 , srv1010001808:90 , srv1011120557:90 , srv1037042113:90 , srv112524859:90 , srv1180264461:90 , srv1311817421:90 , srv1354127973:90 , srv1443783857:90 , srv1483015092:90 , srv1610517590:90 , srv1669883417:90 , srv1857222085:90 , srv2007387437:90 , srv2033844707:90 , srv2057312481:90 , srv2080204621:90 , srv249886313:90 , srv332826375:90 , srv354081565:90 , srv459559218:90 , srv501411083:90 , srv53828647:90 , srv668843480:90 , srv675783815:90 , srv705530819:90 , srv806282589:90 , srv912789701:90 , srv950444858:90 , srv975452874:90 }
2023-05-30 23:55:28,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-30 23:55:28,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2057312481=15, srv1010001808=1, srv332826375=18, srv806282589=26, srv501411083=21, srv950444858=28, srv1857222085=12, srv1180264461=5, srv912789701=27, srv2007387437=13, srv2033844707=14, srv1311817421=6, srv705530819=25, srv1011120557=2, srv459559218=20, srv1483015092=9, srv975452874=29, srv1009056219=0, srv249886313=17, srv2080204621=16, srv675783815=24, srv1037042113=3, srv1610517590=10, srv53828647=22, srv112524859=4, srv1443783857=8, srv1669883417=11, srv668843480=23, srv1354127973=7, srv354081565=19} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-30 23:55:28,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-30 23:55:28,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-30 23:55:28,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-30 23:55:28,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-30 23:55:28,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-30 23:55:28,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-30 23:55:28,879 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-30 23:55:28,879 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1009056219:90 , srv1010001808:90 , srv1011120557:90 , srv1037042113:90 , srv112524859:90 , srv1180264461:90 , srv1311817421:90 , srv1354127973:90 , srv1443783857:90 , srv1483015092:90 , srv1610517590:90 , srv1669883417:90 , srv1857222085:90 , srv2007387437:90 , srv2033844707:90 , srv2057312481:90 , srv2080204621:90 , srv249886313:90 , srv332826375:90 , srv354081565:90 , srv459559218:90 , srv501411083:90 , srv53828647:90 , srv668843480:90 , srv675783815:90 , srv705530819:90 , srv806282589:90 , srv912789701:90 , srv950444858:90 , srv975452874:90 }
2023-05-30 23:55:28,892 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=275 (was 298), ProcessCount=170 (was 170), AvailableMemoryMB=3223 (was 3324)