2023-10-27 06:53:27,674 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-27 06:53:27,999 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-27 06:53:28,126 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-27 06:53:28,216 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-27 06:53:28,226 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-27 06:53:28,241 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=169, AvailableMemoryMB=10627
2023-10-27 06:53:28,243 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-27 06:53:28,244 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-27 06:53:28,381 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000767812:84 , srv1015749769:84 , srv1038812280:84 , srv1055652174:84 , srv1099180420:84 , srv1112629914:84 , srv118078449:84 , srv118603972:84 , srv1192144667:84 , srv1208196391:84 , srv1229960569:84 , srv1239250201:84 , srv1264382105:84 , srv1267366248:84 , srv1286679997:84 , srv1287330636:84 , srv1288659112:84 , srv1333136935:84 , srv1334169942:84 , srv136338665:84 , srv1387598724:84 , srv1400247801:84 , srv1406609664:84 , srv1414892014:84 , srv1435035452:84 , srv1435116750:84 , srv1458557836:84 , srv148006398:84 , srv1483319832:84 , srv1497630384:84 , srv1502703193:84 , srv1519736070:84 , srv1536478760:84 , srv1548517233:84 , srv1605322021:84 , srv1632043795:84 , srv1645198978:84 , srv1645818917:84 , srv1650788887:84 , srv1678676012:84 , srv1687778231:84 , srv1699624716:84 , srv1705297149:84 , srv1719441108:84 , srv1742031221:84 , srv1749008261:84 , srv1827501489:84 , srv1867427595:84 , srv1881408924:84 , srv189101372:84 , srv1901283426:84 , srv1915755537:84 , srv1923286578:84 , srv194189900:84 , srv1962004696:84 , srv1963629935:84 , srv1987586780:84 , srv2000017288:84 , srv2003639987:84 , srv201997636:84 , srv2079626066:84 , srv2082671353:84 , srv2136362000:84 , srv2145482821:84 , srv228014442:84 , srv246221677:84 , srv324800351:84 , srv336928822:84 , srv340477205:84 , srv415963093:84 , srv430371021:84 , srv448625199:84 , srv473156100:84 , srv4807686:84 , srv537705647:84 , srv549734603:84 , srv574798841:84 , srv575086955:84 , srv587317810:84 , srv600684446:84 , srv631420446:84 , srv633598939:84 , srv646835921:84 , srv652506160:84 , srv654918830:84 , srv655450713:84 , srv665323462:84 , srv684230577:84 , srv726156187:84 , srv729524144:84 , srv755029355:84 , srv757510351:84 , srv786230992:84 , srv815836563:84 , srv837824412:84 , srv857027894:84 , srv917585587:84 , srv926778304:84 , srv947361417:84 , srv467680795:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-27 06:53:28,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:28,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1923286578=52, srv189101372=49, srv1548517233=33, srv118078449=6, srv1333136935=17, srv837824412=95, srv1287330636=15, srv1719441108=43, srv633598939=82, srv1208196391=9, srv1881408924=48, srv2003639987=58, srv1192144667=8, srv4807686=74, srv1055652174=3, srv1915755537=51, srv1099180420=4, srv1112629914=5, srv537705647=75, srv430371021=70, srv136338665=19, srv1406609664=22, srv757510351=92, srv1400247801=21, srv1901283426=50, srv1435116750=25, srv1387598724=20, srv600684446=80, srv1687778231=40, srv1288659112=16, srv1502703193=30, srv857027894=96, srv1742031221=44, srv201997636=59, srv340477205=68, srv473156100=73, srv1334169942=18, srv917585587=97, srv1414892014=23, srv148006398=27, srv1699624716=41, srv1827501489=46, srv448625199=71, srv415963093=69, srv2136362000=62, srv1632043795=35, srv631420446=81, srv246221677=65, srv1483319832=28, srv1038812280=2, srv2082671353=61, srv574798841=77, srv786230992=93, srv652506160=84, srv228014442=64, srv118603972=7, srv1497630384=29, srv1963629935=55, srv654918830=85, srv1435035452=24, srv1749008261=45, srv1536478760=32, srv324800351=66, srv1267366248=13, srv1867427595=47, srv1650788887=38, srv1458557836=26, srv684230577=88, srv665323462=87, srv549734603=76, srv336928822=67, srv2000017288=57, srv1264382105=12, srv729524144=90, srv1286679997=14, srv1987586780=56, srv2079626066=60, srv755029355=91, srv1239250201=11, srv1645198978=36, srv926778304=98, srv1519736070=31, srv646835921=83, srv587317810=79, srv815836563=94, srv947361417=99, srv1605322021=34, srv655450713=86, srv726156187=89, srv1645818917=37, srv1705297149=42, srv1000767812=0, srv1015749769=1, srv2145482821=63, srv1229960569=10, srv575086955=78, srv467680795=72, srv194189900=53, srv1962004696=54, srv1678676012=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:28,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:28,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:28,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:28,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-27 06:53:28,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-27 06:53:28,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-27 06:53:28,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-27 06:53:28,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-27 06:53:28,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-27 06:53:28,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-27 06:53:28,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-27 06:53:28,677 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:28,677 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-27 06:53:28,678 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-27 06:53:29,099 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 593 ms to try 5000 different iterations.  Found a solution that moves 4746 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17242145446234702. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039393392611820205, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05791553120026401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5273333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039393392611820205, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16666666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24316666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:29,114 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv228014442:81 , srv537705647:81 , srv684230577:81 , srv1287330636:82 , srv1333136935:82 , srv1400247801:82 , srv655450713:82 , srv755029355:82 , srv1015749769:83 , srv118603972:83 , srv1742031221:83 , srv1749008261:83 , srv1867427595:83 , srv448625199:83 , srv1229960569:84 , srv1334169942:84 , srv1605322021:84 , srv1915755537:84 , srv1962004696:84 , srv1963629935:84 , srv654918830:84 , srv786230992:84 , srv1000767812:85 , srv1192144667:85 , srv1239250201:85 , srv136338665:85 , srv1414892014:85 , srv1483319832:85 , srv1502703193:85 , srv1645198978:85 , srv1705297149:85 , srv2003639987:85 , srv574798841:85 , srv857027894:85 , srv947361417:85 , srv1536478760:86 , srv1645818917:86 , srv1687778231:86 , srv1827501489:86 , srv1881408924:86 , srv1901283426:86 , srv2079626066:86 , srv324800351:86 , srv473156100:86 , srv549734603:86 , srv587317810:86 , srv600684446:86 , srv665323462:86 , srv757510351:86 , srv1099180420:87 , srv1286679997:87 , srv1288659112:87 , srv1435116750:87 , srv148006398:87 , srv1519736070:87 , srv1548517233:87 , srv1632043795:87 , srv1650788887:87 , srv1678676012:87 , srv1699624716:87 , srv189101372:87 , srv2136362000:87 , srv430371021:87 , srv631420446:87 , srv633598939:87 , srv729524144:87 , srv815836563:87 , srv917585587:87 , srv1112629914:88 , srv1406609664:88 , srv1923286578:88 , srv2082671353:88 , srv246221677:88 , srv340477205:88 , srv646835921:88 , srv726156187:88 , srv837824412:88 , srv1055652174:89 , srv118078449:89 , srv1497630384:89 , srv1987586780:89 , srv2000017288:89 , srv2145482821:89 , srv415963093:89 , srv4807686:89 , srv926778304:89 , srv1038812280:90 , srv1208196391:90 , srv1387598724:90 , srv1458557836:90 , srv1719441108:90 , srv194189900:90 , srv575086955:90 , srv652506160:90 , srv1267366248:91 , srv336928822:92 , srv1264382105:93 , srv1435035452:93 , srv201997636:93 , srv467680795:440 }
2023-10-27 06:53:29,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:29,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1923286578=52, srv189101372=49, srv1548517233=33, srv118078449=6, srv1333136935=17, srv837824412=95, srv1287330636=15, srv1719441108=43, srv633598939=82, srv1208196391=9, srv1881408924=48, srv2003639987=58, srv1192144667=8, srv4807686=74, srv1055652174=3, srv1915755537=51, srv1099180420=4, srv1112629914=5, srv537705647=75, srv430371021=70, srv136338665=19, srv1406609664=22, srv757510351=92, srv1400247801=21, srv1901283426=50, srv1435116750=25, srv1387598724=20, srv600684446=80, srv1687778231=40, srv1288659112=16, srv1502703193=30, srv857027894=96, srv1742031221=44, srv201997636=59, srv340477205=68, srv473156100=73, srv1334169942=18, srv917585587=97, srv1414892014=23, srv148006398=27, srv1699624716=41, srv1827501489=46, srv448625199=71, srv415963093=69, srv2136362000=62, srv1632043795=35, srv631420446=81, srv246221677=65, srv1483319832=28, srv1038812280=2, srv2082671353=61, srv574798841=77, srv786230992=93, srv652506160=84, srv228014442=64, srv118603972=7, srv1497630384=29, srv1963629935=55, srv654918830=85, srv1435035452=24, srv1749008261=45, srv1536478760=32, srv324800351=66, srv1267366248=13, srv1867427595=47, srv1650788887=38, srv1458557836=26, srv684230577=88, srv665323462=87, srv549734603=76, srv336928822=67, srv2000017288=57, srv1264382105=12, srv729524144=90, srv1286679997=14, srv1987586780=56, srv2079626066=60, srv755029355=91, srv1239250201=11, srv1645198978=36, srv926778304=98, srv1519736070=31, srv646835921=83, srv587317810=79, srv815836563=94, srv947361417=99, srv1605322021=34, srv655450713=86, srv726156187=89, srv1645818917=37, srv1705297149=42, srv1000767812=0, srv1015749769=1, srv2145482821=63, srv1229960569=10, srv575086955=78, srv467680795=72, srv194189900=53, srv1962004696=54, srv1678676012=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:29,131 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:29,132 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:29,133 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-27 06:53:29,134 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-27 06:53:29,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-27 06:53:29,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-27 06:53:29,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-27 06:53:29,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-27 06:53:29,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-27 06:53:29,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039393392611820205, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05791553120026401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039393392611820205, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16666666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24316666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:29,257 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-27 06:53:29,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17238822226006092, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039393392611820205, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05791553120026401, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039393392611820205, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16666666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24316666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-27 06:53:29,554 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 399 ms to try 5000 different iterations.  Found a solution that moves 2646 regions; Going from a computed imbalance of 0.17238822226006092 to a new imbalance of 0.015846575928642127. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01869678506567356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03125866883129186, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.294, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01869678506567356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:29,563 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1055652174:80 , srv1435035452:80 , srv1458557836:80 , srv201997636:80 , srv336928822:80 , srv1267366248:81 , srv575086955:81 , srv1264382105:82 , srv1387598724:82 , srv646835921:82 , srv1112629914:83 , srv118078449:83 , srv1208196391:83 , srv1286679997:83 , srv1406609664:83 , srv1497630384:83 , srv1632043795:83 , srv1719441108:83 , srv194189900:83 , srv2082671353:83 , srv415963093:83 , srv4807686:83 , srv652506160:83 , srv926778304:83 , srv1435116750:84 , srv1548517233:84 , srv1687778231:84 , srv340477205:84 , srv726156187:84 , srv815836563:84 , srv917585587:84 , srv1038812280:85 , srv1519736070:85 , srv1650788887:85 , srv1881408924:85 , srv189101372:85 , srv1923286578:85 , srv1987586780:85 , srv2000017288:85 , srv2136362000:85 , srv246221677:85 , srv430371021:85 , srv473156100:85 , srv633598939:85 , srv757510351:85 , srv1099180420:86 , srv148006398:86 , srv1605322021:86 , srv1699624716:86 , srv1901283426:86 , srv2003639987:86 , srv2145482821:86 , srv587317810:86 , srv631420446:86 , srv729524144:86 , srv786230992:86 , srv837824412:86 , srv857027894:86 , srv118603972:87 , srv1192144667:87 , srv1239250201:87 , srv1288659112:87 , srv1536478760:87 , srv1678676012:87 , srv1827501489:87 , srv324800351:87 , srv549734603:87 , srv654918830:87 , srv665323462:87 , srv1015749769:88 , srv1333136935:88 , srv1400247801:88 , srv1414892014:88 , srv1502703193:88 , srv1705297149:88 , srv1867427595:88 , srv1962004696:88 , srv1963629935:88 , srv574798841:88 , srv600684446:88 , srv1000767812:89 , srv1229960569:89 , srv1334169942:89 , srv136338665:89 , srv1483319832:89 , srv1645198978:89 , srv1645818917:89 , srv1749008261:89 , srv2079626066:89 , srv228014442:89 , srv947361417:89 , srv1287330636:90 , srv1915755537:90 , srv1742031221:91 , srv448625199:91 , srv655450713:91 , srv684230577:91 , srv755029355:92 , srv537705647:94 , srv467680795:500 }
2023-10-27 06:53:29,566 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:29,567 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1923286578=52, srv189101372=49, srv1548517233=33, srv118078449=6, srv1333136935=17, srv837824412=95, srv1287330636=15, srv1719441108=43, srv633598939=82, srv1208196391=9, srv1881408924=48, srv2003639987=58, srv1192144667=8, srv4807686=74, srv1055652174=3, srv1915755537=51, srv1099180420=4, srv1112629914=5, srv537705647=75, srv430371021=70, srv136338665=19, srv1406609664=22, srv757510351=92, srv1400247801=21, srv1901283426=50, srv1435116750=25, srv1387598724=20, srv600684446=80, srv1687778231=40, srv1288659112=16, srv1502703193=30, srv857027894=96, srv1742031221=44, srv201997636=59, srv340477205=68, srv473156100=73, srv1334169942=18, srv917585587=97, srv1414892014=23, srv148006398=27, srv1699624716=41, srv1827501489=46, srv448625199=71, srv415963093=69, srv2136362000=62, srv1632043795=35, srv631420446=81, srv246221677=65, srv1483319832=28, srv1038812280=2, srv2082671353=61, srv574798841=77, srv786230992=93, srv652506160=84, srv228014442=64, srv118603972=7, srv1497630384=29, srv1963629935=55, srv654918830=85, srv1435035452=24, srv1749008261=45, srv1536478760=32, srv324800351=66, srv1267366248=13, srv1867427595=47, srv1650788887=38, srv1458557836=26, srv684230577=88, srv665323462=87, srv549734603=76, srv336928822=67, srv2000017288=57, srv1264382105=12, srv729524144=90, srv1286679997=14, srv1987586780=56, srv2079626066=60, srv755029355=91, srv1239250201=11, srv1645198978=36, srv926778304=98, srv1519736070=31, srv646835921=83, srv587317810=79, srv815836563=94, srv947361417=99, srv1605322021=34, srv655450713=86, srv726156187=89, srv1645818917=37, srv1705297149=42, srv1000767812=0, srv1015749769=1, srv2145482821=63, srv1229960569=10, srv575086955=78, srv467680795=72, srv194189900=53, srv1962004696=54, srv1678676012=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:29,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:29,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:29,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:29,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:29,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:29,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:29,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:29,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-27 06:53:29,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-27 06:53:29,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-27 06:53:29,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-27 06:53:29,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-27 06:53:29,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-27 06:53:29,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-27 06:53:29,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-27 06:53:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-27 06:53:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-27 06:53:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-27 06:53:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-27 06:53:29,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-27 06:53:29,752 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01869678506567356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03125866883129186, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01869678506567356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:29,752 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-27 06:53:29,752 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01582804824064191, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01869678506567356); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03125866883129186, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01869678506567356); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04491666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-27 06:53:30,076 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 997 regions; Going from a computed imbalance of 0.01582804824064191 to a new imbalance of 5.73073961379418E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009475587393582686); MoveCostFunction : (multiplier=7.0, imbalance=0.11077777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:30,078 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1264382105:82 , srv1645818917:82 , srv2145482821:82 , srv537705647:82 , srv1038812280:83 , srv1288659112:83 , srv136338665:83 , srv1678676012:83 , srv1742031221:83 , srv1915755537:83 , srv2079626066:83 , srv600684446:83 , srv755029355:83 , srv1000767812:84 , srv1414892014:84 , srv1483319832:84 , srv1502703193:84 , srv1645198978:84 , srv1699624716:84 , srv1705297149:84 , srv1827501489:84 , srv1987586780:84 , srv2000017288:84 , srv448625199:84 , srv549734603:84 , srv574798841:84 , srv631420446:84 , srv655450713:84 , srv729524144:84 , srv837824412:84 , srv947361417:84 , srv1099180420:85 , srv118078449:85 , srv1192144667:85 , srv1208196391:85 , srv1229960569:85 , srv1239250201:85 , srv1334169942:85 , srv1387598724:85 , srv1406609664:85 , srv1435035452:85 , srv148006398:85 , srv1497630384:85 , srv1519736070:85 , srv1536478760:85 , srv1650788887:85 , srv1719441108:85 , srv1749008261:85 , srv189101372:85 , srv1901283426:85 , srv1923286578:85 , srv194189900:85 , srv1963629935:85 , srv246221677:85 , srv324800351:85 , srv415963093:85 , srv430371021:85 , srv4807686:85 , srv587317810:85 , srv633598939:85 , srv652506160:85 , srv665323462:85 , srv684230577:85 , srv1112629914:86 , srv1267366248:86 , srv1548517233:86 , srv1605322021:86 , srv1881408924:86 , srv1962004696:86 , srv2003639987:86 , srv201997636:86 , srv2136362000:86 , srv336928822:86 , srv340477205:86 , srv575086955:86 , srv654918830:86 , srv726156187:86 , srv815836563:86 , srv917585587:86 , srv926778304:86 , srv1015749769:87 , srv118603972:87 , srv1286679997:87 , srv1287330636:87 , srv1435116750:87 , srv1458557836:87 , srv1687778231:87 , srv1867427595:87 , srv2082671353:87 , srv646835921:87 , srv757510351:87 , srv857027894:87 , srv1333136935:88 , srv1400247801:88 , srv1632043795:88 , srv228014442:88 , srv473156100:88 , srv786230992:88 , srv1055652174:89 , srv467680795:570 }
2023-10-27 06:53:30,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:30,083 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1923286578=52, srv189101372=49, srv1548517233=33, srv118078449=6, srv1333136935=17, srv837824412=95, srv1287330636=15, srv1719441108=43, srv633598939=82, srv1208196391=9, srv1881408924=48, srv2003639987=58, srv1192144667=8, srv4807686=74, srv1055652174=3, srv1915755537=51, srv1099180420=4, srv1112629914=5, srv537705647=75, srv430371021=70, srv136338665=19, srv1406609664=22, srv757510351=92, srv1400247801=21, srv1901283426=50, srv1435116750=25, srv1387598724=20, srv600684446=80, srv1687778231=40, srv1288659112=16, srv1502703193=30, srv857027894=96, srv1742031221=44, srv201997636=59, srv340477205=68, srv473156100=73, srv1334169942=18, srv917585587=97, srv1414892014=23, srv148006398=27, srv1699624716=41, srv1827501489=46, srv448625199=71, srv415963093=69, srv2136362000=62, srv1632043795=35, srv631420446=81, srv246221677=65, srv1483319832=28, srv1038812280=2, srv2082671353=61, srv574798841=77, srv786230992=93, srv652506160=84, srv228014442=64, srv118603972=7, srv1497630384=29, srv1963629935=55, srv654918830=85, srv1435035452=24, srv1749008261=45, srv1536478760=32, srv324800351=66, srv1267366248=13, srv1867427595=47, srv1650788887=38, srv1458557836=26, srv684230577=88, srv665323462=87, srv549734603=76, srv336928822=67, srv2000017288=57, srv1264382105=12, srv729524144=90, srv1286679997=14, srv1987586780=56, srv2079626066=60, srv755029355=91, srv1239250201=11, srv1645198978=36, srv926778304=98, srv1519736070=31, srv646835921=83, srv587317810=79, srv815836563=94, srv947361417=99, srv1605322021=34, srv655450713=86, srv726156187=89, srv1645818917=37, srv1705297149=42, srv1000767812=0, srv1015749769=1, srv2145482821=63, srv1229960569=10, srv575086955=78, srv467680795=72, srv194189900=53, srv1962004696=54, srv1678676012=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:30,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:30,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-27 06:53:30,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-27 06:53:30,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-27 06:53:30,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-27 06:53:30,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-27 06:53:30,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-27 06:53:30,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-27 06:53:30,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-27 06:53:30,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-27 06:53:30,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-27 06:53:30,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-27 06:53:30,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-27 06:53:30,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-27 06:53:30,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009475587393582686); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:30,238 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-27 06:53:30,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.660928181684523E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00587308087272118); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009475587393582686); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00587308087272118); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-27 06:53:30,662 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 542 ms to try 5000 different iterations.  Found a solution that moves 224 regions; Going from a computed imbalance of 5.660928181684523E-4 to a new imbalance of 1.2381077309963607E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.024888888888888887); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:30,663 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015749769:83 , srv1287330636:83 , srv1483319832:83 , srv1867427595:83 , srv201997636:83 , srv652506160:83 , srv1000767812:84 , srv1038812280:84 , srv1055652174:84 , srv1099180420:84 , srv1208196391:84 , srv1229960569:84 , srv1267366248:84 , srv1333136935:84 , srv1334169942:84 , srv1400247801:84 , srv1435035452:84 , srv148006398:84 , srv1536478760:84 , srv1632043795:84 , srv1645198978:84 , srv1650788887:84 , srv1699624716:84 , srv1719441108:84 , srv1742031221:84 , srv1923286578:84 , srv1962004696:84 , srv1963629935:84 , srv1987586780:84 , srv2000017288:84 , srv2079626066:84 , srv2082671353:84 , srv228014442:84 , srv246221677:84 , srv336928822:84 , srv340477205:84 , srv448625199:84 , srv473156100:84 , srv600684446:84 , srv665323462:84 , srv726156187:84 , srv757510351:84 , srv786230992:84 , srv837824412:84 , srv857027894:84 , srv947361417:84 , srv1112629914:85 , srv118603972:85 , srv1192144667:85 , srv1239250201:85 , srv1264382105:85 , srv1286679997:85 , srv1288659112:85 , srv136338665:85 , srv1387598724:85 , srv1435116750:85 , srv1458557836:85 , srv1497630384:85 , srv1502703193:85 , srv1519736070:85 , srv1548517233:85 , srv1645818917:85 , srv1678676012:85 , srv1687778231:85 , srv1705297149:85 , srv1749008261:85 , srv1827501489:85 , srv1881408924:85 , srv189101372:85 , srv1901283426:85 , srv1915755537:85 , srv194189900:85 , srv2003639987:85 , srv2136362000:85 , srv2145482821:85 , srv324800351:85 , srv415963093:85 , srv430371021:85 , srv4807686:85 , srv537705647:85 , srv549734603:85 , srv574798841:85 , srv575086955:85 , srv587317810:85 , srv631420446:85 , srv646835921:85 , srv654918830:85 , srv655450713:85 , srv684230577:85 , srv729524144:85 , srv755029355:85 , srv815836563:85 , srv917585587:85 , srv926778304:85 , srv118078449:86 , srv1406609664:86 , srv1414892014:86 , srv1605322021:86 , srv633598939:86 , srv467680795:632 }
2023-10-27 06:53:30,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:30,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1923286578=52, srv189101372=49, srv1548517233=33, srv118078449=6, srv1333136935=17, srv837824412=95, srv1287330636=15, srv1719441108=43, srv633598939=82, srv1208196391=9, srv1881408924=48, srv2003639987=58, srv1192144667=8, srv4807686=74, srv1055652174=3, srv1915755537=51, srv1099180420=4, srv1112629914=5, srv537705647=75, srv430371021=70, srv136338665=19, srv1406609664=22, srv757510351=92, srv1400247801=21, srv1901283426=50, srv1435116750=25, srv1387598724=20, srv600684446=80, srv1687778231=40, srv1288659112=16, srv1502703193=30, srv857027894=96, srv1742031221=44, srv201997636=59, srv340477205=68, srv473156100=73, srv1334169942=18, srv917585587=97, srv1414892014=23, srv148006398=27, srv1699624716=41, srv1827501489=46, srv448625199=71, srv415963093=69, srv2136362000=62, srv1632043795=35, srv631420446=81, srv246221677=65, srv1483319832=28, srv1038812280=2, srv2082671353=61, srv574798841=77, srv786230992=93, srv652506160=84, srv228014442=64, srv118603972=7, srv1497630384=29, srv1963629935=55, srv654918830=85, srv1435035452=24, srv1749008261=45, srv1536478760=32, srv324800351=66, srv1267366248=13, srv1867427595=47, srv1650788887=38, srv1458557836=26, srv684230577=88, srv665323462=87, srv549734603=76, srv336928822=67, srv2000017288=57, srv1264382105=12, srv729524144=90, srv1286679997=14, srv1987586780=56, srv2079626066=60, srv755029355=91, srv1239250201=11, srv1645198978=36, srv926778304=98, srv1519736070=31, srv646835921=83, srv587317810=79, srv815836563=94, srv947361417=99, srv1605322021=34, srv655450713=86, srv726156187=89, srv1645818917=37, srv1705297149=42, srv1000767812=0, srv1015749769=1, srv2145482821=63, srv1229960569=10, srv575086955=78, srv467680795=72, srv194189900=53, srv1962004696=54, srv1678676012=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:30,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-27 06:53:30,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-27 06:53:30,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-27 06:53:30,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-27 06:53:30,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-27 06:53:30,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-27 06:53:30,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-27 06:53:30,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-27 06:53:30,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-27 06:53:30,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-27 06:53:30,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-27 06:53:30,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-27 06:53:30,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-27 06:53:30,824 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0812595786135791E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:30,824 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1015749769:83 , srv1287330636:83 , srv1483319832:83 , srv1867427595:83 , srv201997636:83 , srv652506160:83 , srv1000767812:84 , srv1038812280:84 , srv1055652174:84 , srv1099180420:84 , srv1208196391:84 , srv1229960569:84 , srv1267366248:84 , srv1333136935:84 , srv1334169942:84 , srv1400247801:84 , srv1435035452:84 , srv148006398:84 , srv1536478760:84 , srv1632043795:84 , srv1645198978:84 , srv1650788887:84 , srv1699624716:84 , srv1719441108:84 , srv1742031221:84 , srv1923286578:84 , srv1962004696:84 , srv1963629935:84 , srv1987586780:84 , srv2000017288:84 , srv2079626066:84 , srv2082671353:84 , srv228014442:84 , srv246221677:84 , srv336928822:84 , srv340477205:84 , srv448625199:84 , srv473156100:84 , srv600684446:84 , srv665323462:84 , srv726156187:84 , srv757510351:84 , srv786230992:84 , srv837824412:84 , srv857027894:84 , srv947361417:84 , srv1112629914:85 , srv118603972:85 , srv1192144667:85 , srv1239250201:85 , srv1264382105:85 , srv1286679997:85 , srv1288659112:85 , srv136338665:85 , srv1387598724:85 , srv1435116750:85 , srv1458557836:85 , srv1497630384:85 , srv1502703193:85 , srv1519736070:85 , srv1548517233:85 , srv1645818917:85 , srv1678676012:85 , srv1687778231:85 , srv1705297149:85 , srv1749008261:85 , srv1827501489:85 , srv1881408924:85 , srv189101372:85 , srv1901283426:85 , srv1915755537:85 , srv194189900:85 , srv2003639987:85 , srv2136362000:85 , srv2145482821:85 , srv324800351:85 , srv415963093:85 , srv430371021:85 , srv4807686:85 , srv537705647:85 , srv549734603:85 , srv574798841:85 , srv575086955:85 , srv587317810:85 , srv631420446:85 , srv646835921:85 , srv654918830:85 , srv655450713:85 , srv684230577:85 , srv729524144:85 , srv755029355:85 , srv815836563:85 , srv917585587:85 , srv926778304:85 , srv118078449:86 , srv1406609664:86 , srv1414892014:86 , srv1605322021:86 , srv633598939:86 , srv467680795:632 }
2023-10-27 06:53:30,851 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=329 (was 279) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=10125 (was 10627)
2023-10-27 06:53:30,861 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=329, ProcessCount=169, AvailableMemoryMB=10125
2023-10-27 06:53:30,862 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-27 06:53:30,862 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-27 06:53:30,870 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1081744014:84 , srv1184487174:84 , srv122767296:84 , srv1380658856:84 , srv1385063617:84 , srv1385777289:84 , srv1464712323:84 , srv1493205712:84 , srv1519382471:84 , srv1521327171:84 , srv1531907631:84 , srv1539222508:84 , srv1595624145:84 , srv1610264092:84 , srv1660267779:84 , srv1966193750:84 , srv1981642733:84 , srv1986500284:84 , srv226378549:84 , srv228962154:84 , srv291149150:84 , srv412379853:84 , srv496497429:84 , srv545959504:84 , srv644103410:84 , srv669962698:84 , srv776682321:84 , srv863591896:84 , srv95500044:84 , srv2007950548:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-27 06:53:30,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:53:30,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv412379853=22, srv545959504=24, srv122767296=2, srv1531907631=10, srv1464712323=6, srv1966193750=15, srv1493205712=7, srv226378549=19, srv228962154=20, srv1595624145=12, srv95500044=29, srv1380658856=3, srv1610264092=13, srv1385063617=4, srv1986500284=17, srv1981642733=16, srv2007950548=18, srv1081744014=0, srv1521327171=9, srv669962698=26, srv1385777289=5, srv1519382471=8, srv1539222508=11, srv291149150=21, srv496497429=23, srv1660267779=14, srv1184487174=1, srv644103410=25, srv776682321=27, srv863591896=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:53:30,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:53:30,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-27 06:53:30,896 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1371852534212608, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:53:30,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912625309518299, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1371852534212608, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-27 06:55:30,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5554824 different iterations.  Found a solution that moves 1851 regions; Going from a computed imbalance of 0.9912625309518299 to a new imbalance of 4.320326340186438E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6855555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:55:30,891 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1081744014:90 , srv1184487174:90 , srv122767296:90 , srv1380658856:90 , srv1385063617:90 , srv1385777289:90 , srv1464712323:90 , srv1493205712:90 , srv1519382471:90 , srv1521327171:90 , srv1531907631:90 , srv1539222508:90 , srv1595624145:90 , srv1610264092:90 , srv1660267779:90 , srv1966193750:90 , srv1981642733:90 , srv1986500284:90 , srv2007950548:90 , srv226378549:90 , srv228962154:90 , srv291149150:90 , srv412379853:90 , srv496497429:90 , srv545959504:90 , srv644103410:90 , srv669962698:90 , srv776682321:90 , srv863591896:90 , srv95500044:90 }
2023-10-27 06:55:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-27 06:55:30,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv412379853=22, srv545959504=24, srv122767296=2, srv1531907631=10, srv1464712323=6, srv1966193750=15, srv1493205712=7, srv226378549=19, srv228962154=20, srv1595624145=12, srv95500044=29, srv1380658856=3, srv1610264092=13, srv1385063617=4, srv1986500284=17, srv1981642733=16, srv2007950548=18, srv1081744014=0, srv1521327171=9, srv669962698=26, srv1385777289=5, srv1519382471=8, srv1539222508=11, srv291149150=21, srv496497429=23, srv1660267779=14, srv1184487174=1, srv644103410=25, srv776682321=27, srv863591896=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-27 06:55:30,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-27 06:55:30,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-27 06:55:30,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-27 06:55:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-27 06:55:30,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-27 06:55:30,912 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-27 06:55:30,912 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1081744014:90 , srv1184487174:90 , srv122767296:90 , srv1380658856:90 , srv1385063617:90 , srv1385777289:90 , srv1464712323:90 , srv1493205712:90 , srv1519382471:90 , srv1521327171:90 , srv1531907631:90 , srv1539222508:90 , srv1595624145:90 , srv1610264092:90 , srv1660267779:90 , srv1966193750:90 , srv1981642733:90 , srv1986500284:90 , srv2007950548:90 , srv226378549:90 , srv228962154:90 , srv291149150:90 , srv412379853:90 , srv496497429:90 , srv545959504:90 , srv644103410:90 , srv669962698:90 , srv776682321:90 , srv863591896:90 , srv95500044:90 }
2023-10-27 06:55:30,928 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 329), ProcessCount=166 (was 169), AvailableMemoryMB=10993 (was 10125) - AvailableMemoryMB LEAK? -