2023-11-11 04:55:15,631 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-11 04:55:15,915 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-11 04:55:16,020 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-11 04:55:16,091 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 04:55:16,098 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-11 04:55:16,111 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=200, ProcessCount=167, AvailableMemoryMB=2932
2023-11-11 04:55:16,113 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 04:55:16,113 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-11 04:55:16,247 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100409858:84 , srv104226524:84 , srv110395153:84 , srv1104174171:84 , srv1109853382:84 , srv112096380:84 , srv1121627461:84 , srv1184481417:84 , srv1194905691:84 , srv1248941656:84 , srv1255959885:84 , srv1259572104:84 , srv1273803934:84 , srv1297589107:84 , srv1300865843:84 , srv1302066959:84 , srv1306980690:84 , srv1333421896:84 , srv1365706583:84 , srv1378246477:84 , srv1390759139:84 , srv1394910569:84 , srv1398025435:84 , srv1440508619:84 , srv1445796511:84 , srv1455579219:84 , srv1471379363:84 , srv1478762778:84 , srv1480098103:84 , srv1485555977:84 , srv1488932493:84 , srv1514373971:84 , srv1534293359:84 , srv1551668639:84 , srv1581592896:84 , srv1605405473:84 , srv167393982:84 , srv1726185166:84 , srv1783502614:84 , srv178634301:84 , srv1800411261:84 , srv1803766102:84 , srv1813569170:84 , srv1827084454:84 , srv1832540137:84 , srv1834391999:84 , srv1848614216:84 , srv1849610312:84 , srv1866343594:84 , srv1868218152:84 , srv1869290169:84 , srv1918959452:84 , srv1959352068:84 , srv1971150344:84 , srv1990354980:84 , srv2041638469:84 , srv2042544588:84 , srv2052032218:84 , srv2064174110:84 , srv2070300908:84 , srv2092112688:84 , srv212233524:84 , srv2133828072:84 , srv2137680770:84 , srv235049208:84 , srv238467075:84 , srv249724131:84 , srv288377538:84 , srv294717339:84 , srv342012692:84 , srv361681285:84 , srv369559327:84 , srv372979765:84 , srv389220266:84 , srv390471424:84 , srv399147516:84 , srv406064715:84 , srv458698055:84 , srv460651088:84 , srv490720432:84 , srv52162442:84 , srv562906435:84 , srv566140504:84 , srv592188157:84 , srv64675221:84 , srv664673092:84 , srv682617228:84 , srv700189506:84 , srv719234457:84 , srv759506360:84 , srv766958575:84 , srv773148327:84 , srv843712502:84 , srv863221191:84 , srv896824458:84 , srv925497113:84 , srv961435309:84 , srv966106435:84 , srv976279726:84 , srv1208005277:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-11 04:55:16,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:16,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv566140504=83, srv966106435=98, srv896824458=95, srv843712502=93, srv1485555977=30, srv1534293359=33, srv1255959885=11, srv389220266=74, srv1803766102=42, srv1184481417=7, srv773148327=92, srv766958575=91, srv1259572104=12, srv1971150344=54, srv1551668639=34, srv2092112688=61, srv1273803934=13, srv682617228=87, srv1398025435=23, srv1194905691=8, srv1297589107=14, srv562906435=82, srv1848614216=47, srv961435309=97, srv361681285=71, srv1918959452=52, srv1478762778=28, srv369559327=72, srv2041638469=56, srv249724131=67, srv212233524=62, srv976279726=99, srv1306980690=17, srv2133828072=63, srv759506360=90, srv178634301=40, srv238467075=66, srv104226524=1, srv1832540137=45, srv863221191=94, srv1488932493=31, srv342012692=70, srv294717339=69, srv399147516=76, srv2042544588=57, srv1480098103=29, srv1959352068=53, srv1581592896=35, srv52162442=81, srv1866343594=49, srv700189506=88, srv1208005277=9, srv1849610312=48, srv64675221=85, srv458698055=78, srv1869290169=51, srv1471379363=27, srv235049208=65, srv664673092=86, srv1800411261=41, srv1300865843=15, srv1455579219=26, srv167393982=37, srv1302066959=16, srv112096380=5, srv406064715=77, srv110395153=2, srv2064174110=59, srv1378246477=20, srv460651088=79, srv1445796511=25, srv2137680770=64, srv1390759139=21, srv2052032218=58, srv1990354980=55, srv1868218152=50, srv390471424=75, srv1394910569=22, srv490720432=80, srv1104174171=3, srv1333421896=18, srv1248941656=10, srv1726185166=38, srv1109853382=4, srv2070300908=60, srv288377538=68, srv719234457=89, srv1605405473=36, srv1834391999=46, srv592188157=84, srv1783502614=39, srv372979765=73, srv100409858=0, srv1813569170=43, srv1440508619=24, srv1827084454=44, srv1121627461=6, srv1514373971=32, srv1365706583=19, srv925497113=96} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:16,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:16,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:16,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:16,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:16,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 04:55:16,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 04:55:16,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 04:55:16,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 04:55:16,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 04:55:16,381 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 04:55:16,382 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 04:55:16,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 04:55:16,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 04:55:16,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 04:55:16,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 04:55:16,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 04:55:16,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 04:55:16,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 04:55:16,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 04:55:16,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 04:55:16,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 04:55:16,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 04:55:16,566 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:16,567 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 04:55:16,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 04:55:17,244 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 825 ms to try 5000 different iterations.  Found a solution that moves 4676 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17848360776691208. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03991948499735651, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055859773125636544, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5195555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03991948499735651, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:17,256 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv100409858:82 , srv1255959885:82 , srv562906435:82 , srv1273803934:83 , srv1455579219:83 , srv1551668639:83 , srv1868218152:83 , srv342012692:83 , srv361681285:83 , srv460651088:83 , srv961435309:83 , srv1333421896:84 , srv1390759139:84 , srv1581592896:84 , srv1783502614:84 , srv1971150344:84 , srv238467075:84 , srv288377538:84 , srv369559327:84 , srv399147516:84 , srv843712502:84 , srv110395153:85 , srv1109853382:85 , srv1194905691:85 , srv1440508619:85 , srv1445796511:85 , srv1478762778:85 , srv566140504:85 , srv863221191:85 , srv925497113:85 , srv1104174171:86 , srv1302066959:86 , srv1365706583:86 , srv1534293359:86 , srv1605405473:86 , srv167393982:86 , srv1834391999:86 , srv1848614216:86 , srv1866343594:86 , srv2041638469:86 , srv2064174110:86 , srv2133828072:86 , srv2137680770:86 , srv235049208:86 , srv294717339:86 , srv406064715:86 , srv458698055:86 , srv700189506:86 , srv719234457:86 , srv759506360:86 , srv773148327:86 , srv104226524:87 , srv1121627461:87 , srv1297589107:87 , srv1306980690:87 , srv1394910569:87 , srv1480098103:87 , srv1514373971:87 , srv1726185166:87 , srv1849610312:87 , srv1869290169:87 , srv1959352068:87 , srv2052032218:87 , srv2092112688:87 , srv52162442:87 , srv64675221:87 , srv664673092:87 , srv682617228:87 , srv766958575:87 , srv966106435:87 , srv1184481417:88 , srv1248941656:88 , srv1398025435:88 , srv178634301:88 , srv1813569170:88 , srv1827084454:88 , srv2070300908:88 , srv372979765:88 , srv389220266:88 , srv390471424:88 , srv490720432:88 , srv896824458:88 , srv976279726:88 , srv112096380:89 , srv1259572104:89 , srv1300865843:89 , srv1485555977:89 , srv1800411261:89 , srv1803766102:89 , srv1990354980:89 , srv212233524:89 , srv592188157:89 , srv1378246477:90 , srv1488932493:90 , srv1832540137:90 , srv2042544588:90 , srv249724131:90 , srv1471379363:91 , srv1918959452:95 , srv1208005277:445 }
2023-11-11 04:55:17,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:17,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv566140504=83, srv966106435=98, srv896824458=95, srv843712502=93, srv1485555977=30, srv1534293359=33, srv1255959885=11, srv389220266=74, srv1803766102=42, srv1184481417=7, srv773148327=92, srv766958575=91, srv1259572104=12, srv1971150344=54, srv1551668639=34, srv2092112688=61, srv1273803934=13, srv682617228=87, srv1398025435=23, srv1194905691=8, srv1297589107=14, srv562906435=82, srv1848614216=47, srv961435309=97, srv361681285=71, srv1918959452=52, srv1478762778=28, srv369559327=72, srv2041638469=56, srv249724131=67, srv212233524=62, srv976279726=99, srv1306980690=17, srv2133828072=63, srv759506360=90, srv178634301=40, srv238467075=66, srv104226524=1, srv1832540137=45, srv863221191=94, srv1488932493=31, srv342012692=70, srv294717339=69, srv399147516=76, srv2042544588=57, srv1480098103=29, srv1959352068=53, srv1581592896=35, srv52162442=81, srv1866343594=49, srv700189506=88, srv1208005277=9, srv1849610312=48, srv64675221=85, srv458698055=78, srv1869290169=51, srv1471379363=27, srv235049208=65, srv664673092=86, srv1800411261=41, srv1300865843=15, srv1455579219=26, srv167393982=37, srv1302066959=16, srv112096380=5, srv406064715=77, srv110395153=2, srv2064174110=59, srv1378246477=20, srv460651088=79, srv1445796511=25, srv2137680770=64, srv1390759139=21, srv2052032218=58, srv1990354980=55, srv1868218152=50, srv390471424=75, srv1394910569=22, srv490720432=80, srv1104174171=3, srv1333421896=18, srv1248941656=10, srv1726185166=38, srv1109853382=4, srv2070300908=60, srv288377538=68, srv719234457=89, srv1605405473=36, srv1834391999=46, srv592188157=84, srv1783502614=39, srv372979765=73, srv100409858=0, srv1813569170=43, srv1440508619=24, srv1827084454=44, srv1121627461=6, srv1514373971=32, srv1365706583=19, srv925497113=96} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:17,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:17,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:17,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 04:55:17,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 04:55:17,282 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 04:55:17,283 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 04:55:17,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 04:55:17,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 04:55:17,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 04:55:17,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03991948499735651, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055859773125636544, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03991948499735651, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:17,457 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 04:55:17,457 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17845086571510216, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03991948499735651, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055859773125636544, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03991948499735651, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17425, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 04:55:17,723 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 408 ms to try 5000 different iterations.  Found a solution that moves 2584 regions; Going from a computed imbalance of 0.17845086571510216 to a new imbalance of 0.01571347816441463. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022844268014533542); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2871111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022844268014533542); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:17,732 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1918959452:77 , srv112096380:80 , srv1259572104:80 , srv1378246477:81 , srv1800411261:81 , srv1869290169:81 , srv1471379363:82 , srv1803766102:82 , srv249724131:82 , srv104226524:83 , srv1300865843:83 , srv1488932493:83 , srv1827084454:83 , srv2042544588:83 , srv372979765:83 , srv390471424:83 , srv592188157:83 , srv1121627461:84 , srv1248941656:84 , srv1302066959:84 , srv1365706583:84 , srv1398025435:84 , srv1485555977:84 , srv167393982:84 , srv178634301:84 , srv1959352068:84 , srv1990354980:84 , srv2052032218:84 , srv294717339:84 , srv52162442:84 , srv664673092:84 , srv700189506:84 , srv896824458:84 , srv976279726:84 , srv1194905691:85 , srv1297589107:85 , srv1514373971:85 , srv1581592896:85 , srv1813569170:85 , srv1832540137:85 , srv1834391999:85 , srv1849610312:85 , srv1866343594:85 , srv2070300908:85 , srv212233524:85 , srv2137680770:85 , srv458698055:85 , srv490720432:85 , srv766958575:85 , srv773148327:85 , srv966106435:85 , srv110395153:86 , srv1109853382:86 , srv1184481417:86 , srv1333421896:86 , srv1394910569:86 , srv1440508619:86 , srv1478762778:86 , srv1605405473:86 , srv2133828072:86 , srv682617228:86 , srv719234457:86 , srv925497113:86 , srv1445796511:87 , srv1534293359:87 , srv2064174110:87 , srv2092112688:87 , srv235049208:87 , srv369559327:87 , srv389220266:87 , srv566140504:87 , srv64675221:87 , srv759506360:87 , srv843712502:87 , srv863221191:87 , srv1104174171:88 , srv1273803934:88 , srv1306980690:88 , srv1726185166:88 , srv1848614216:88 , srv2041638469:88 , srv288377538:88 , srv342012692:88 , srv361681285:88 , srv100409858:89 , srv1255959885:89 , srv1455579219:89 , srv1480098103:89 , srv1551668639:89 , srv1783502614:89 , srv1868218152:89 , srv1971150344:89 , srv238467075:89 , srv399147516:89 , srv406064715:89 , srv460651088:89 , srv562906435:89 , srv961435309:89 , srv1390759139:91 , srv1208005277:532 }
2023-11-11 04:55:17,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:17,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv566140504=83, srv966106435=98, srv896824458=95, srv843712502=93, srv1485555977=30, srv1534293359=33, srv1255959885=11, srv389220266=74, srv1803766102=42, srv1184481417=7, srv773148327=92, srv766958575=91, srv1259572104=12, srv1971150344=54, srv1551668639=34, srv2092112688=61, srv1273803934=13, srv682617228=87, srv1398025435=23, srv1194905691=8, srv1297589107=14, srv562906435=82, srv1848614216=47, srv961435309=97, srv361681285=71, srv1918959452=52, srv1478762778=28, srv369559327=72, srv2041638469=56, srv249724131=67, srv212233524=62, srv976279726=99, srv1306980690=17, srv2133828072=63, srv759506360=90, srv178634301=40, srv238467075=66, srv104226524=1, srv1832540137=45, srv863221191=94, srv1488932493=31, srv342012692=70, srv294717339=69, srv399147516=76, srv2042544588=57, srv1480098103=29, srv1959352068=53, srv1581592896=35, srv52162442=81, srv1866343594=49, srv700189506=88, srv1208005277=9, srv1849610312=48, srv64675221=85, srv458698055=78, srv1869290169=51, srv1471379363=27, srv235049208=65, srv664673092=86, srv1800411261=41, srv1300865843=15, srv1455579219=26, srv167393982=37, srv1302066959=16, srv112096380=5, srv406064715=77, srv110395153=2, srv2064174110=59, srv1378246477=20, srv460651088=79, srv1445796511=25, srv2137680770=64, srv1390759139=21, srv2052032218=58, srv1990354980=55, srv1868218152=50, srv390471424=75, srv1394910569=22, srv490720432=80, srv1104174171=3, srv1333421896=18, srv1248941656=10, srv1726185166=38, srv1109853382=4, srv2070300908=60, srv288377538=68, srv719234457=89, srv1605405473=36, srv1834391999=46, srv592188157=84, srv1783502614=39, srv372979765=73, srv100409858=0, srv1813569170=43, srv1440508619=24, srv1827084454=44, srv1121627461=6, srv1514373971=32, srv1365706583=19, srv925497113=96} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:17,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:17,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:17,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:17,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:17,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:17,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 04:55:17,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 04:55:17,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 04:55:17,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 04:55:17,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 04:55:17,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 04:55:17,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022844268014533542); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022844268014533542); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:17,875 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 04:55:17,876 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01569538460969333, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022844268014533542); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022844268014533542); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 04:55:18,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 356 ms to try 5000 different iterations.  Found a solution that moves 983 regions; Going from a computed imbalance of 0.01569538460969333 to a new imbalance of 0.0011138116209963211. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006436350813697321); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00735503370238284); MoveCostFunction : (multiplier=7.0, imbalance=0.10922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006436350813697321); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:18,125 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1480098103:80 , srv1306980690:82 , srv1832540137:82 , srv389220266:82 , srv1104174171:83 , srv1390759139:83 , srv1726185166:83 , srv235049208:83 , srv406064715:83 , srv1184481417:84 , srv1488932493:84 , srv1534293359:84 , srv1848614216:84 , srv1971150344:84 , srv2041638469:84 , srv2042544588:84 , srv2092112688:84 , srv212233524:84 , srv238467075:84 , srv490720432:84 , srv682617228:84 , srv1248941656:85 , srv1440508619:85 , srv1485555977:85 , srv1551668639:85 , srv178634301:85 , srv1813569170:85 , srv1918959452:85 , srv1990354980:85 , srv2064174110:85 , srv2070300908:85 , srv249724131:85 , srv288377538:85 , srv399147516:85 , srv460651088:85 , srv52162442:85 , srv566140504:85 , srv592188157:85 , srv64675221:85 , srv719234457:85 , srv759506360:85 , srv766958575:85 , srv896824458:85 , srv961435309:85 , srv966106435:85 , srv100409858:86 , srv110395153:86 , srv1109853382:86 , srv1121627461:86 , srv1273803934:86 , srv1297589107:86 , srv1300865843:86 , srv1394910569:86 , srv1398025435:86 , srv1445796511:86 , srv1455579219:86 , srv1471379363:86 , srv1478762778:86 , srv1514373971:86 , srv1605405473:86 , srv1783502614:86 , srv1803766102:86 , srv1827084454:86 , srv1849610312:86 , srv1866343594:86 , srv1868218152:86 , srv1959352068:86 , srv2052032218:86 , srv2133828072:86 , srv2137680770:86 , srv342012692:86 , srv372979765:86 , srv390471424:86 , srv458698055:86 , srv664673092:86 , srv773148327:86 , srv843712502:86 , srv863221191:86 , srv976279726:86 , srv104226524:87 , srv1194905691:87 , srv1255959885:87 , srv1302066959:87 , srv1333421896:87 , srv1378246477:87 , srv167393982:87 , srv1834391999:87 , srv294717339:87 , srv361681285:87 , srv369559327:87 , srv562906435:87 , srv925497113:87 , srv112096380:88 , srv1259572104:88 , srv1365706583:88 , srv1581592896:88 , srv1800411261:88 , srv700189506:88 , srv1869290169:90 , srv1208005277:538 }
2023-11-11 04:55:18,128 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:18,129 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv566140504=83, srv966106435=98, srv896824458=95, srv843712502=93, srv1485555977=30, srv1534293359=33, srv1255959885=11, srv389220266=74, srv1803766102=42, srv1184481417=7, srv773148327=92, srv766958575=91, srv1259572104=12, srv1971150344=54, srv1551668639=34, srv2092112688=61, srv1273803934=13, srv682617228=87, srv1398025435=23, srv1194905691=8, srv1297589107=14, srv562906435=82, srv1848614216=47, srv961435309=97, srv361681285=71, srv1918959452=52, srv1478762778=28, srv369559327=72, srv2041638469=56, srv249724131=67, srv212233524=62, srv976279726=99, srv1306980690=17, srv2133828072=63, srv759506360=90, srv178634301=40, srv238467075=66, srv104226524=1, srv1832540137=45, srv863221191=94, srv1488932493=31, srv342012692=70, srv294717339=69, srv399147516=76, srv2042544588=57, srv1480098103=29, srv1959352068=53, srv1581592896=35, srv52162442=81, srv1866343594=49, srv700189506=88, srv1208005277=9, srv1849610312=48, srv64675221=85, srv458698055=78, srv1869290169=51, srv1471379363=27, srv235049208=65, srv664673092=86, srv1800411261=41, srv1300865843=15, srv1455579219=26, srv167393982=37, srv1302066959=16, srv112096380=5, srv406064715=77, srv110395153=2, srv2064174110=59, srv1378246477=20, srv460651088=79, srv1445796511=25, srv2137680770=64, srv1390759139=21, srv2052032218=58, srv1990354980=55, srv1868218152=50, srv390471424=75, srv1394910569=22, srv490720432=80, srv1104174171=3, srv1333421896=18, srv1248941656=10, srv1726185166=38, srv1109853382=4, srv2070300908=60, srv288377538=68, srv719234457=89, srv1605405473=36, srv1834391999=46, srv592188157=84, srv1783502614=39, srv372979765=73, srv100409858=0, srv1813569170=43, srv1440508619=24, srv1827084454=44, srv1121627461=6, srv1514373971=32, srv1365706583=19, srv925497113=96} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:18,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:18,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:18,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:18,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 04:55:18,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 04:55:18,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 04:55:18,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 04:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 04:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 04:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 04:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 04:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 04:55:18,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006436350813697321); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00735503370238284); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006436350813697321); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:18,259 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-11 04:55:18,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0011069285078805946, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006436350813697321); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00735503370238284); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006436350813697321); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=7.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-11 04:55:18,584 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 423 ms to try 5000 different iterations.  Found a solution that moves 257 regions; Going from a computed imbalance of 0.0011069285078805946 to a new imbalance of 2.1456368029639813E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.028555555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:18,585 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1394910569:83 , srv64675221:83 , srv1184481417:84 , srv1255959885:84 , srv1273803934:84 , srv1297589107:84 , srv1300865843:84 , srv1365706583:84 , srv1378246477:84 , srv1390759139:84 , srv1398025435:84 , srv1445796511:84 , srv1455579219:84 , srv1471379363:84 , srv1485555977:84 , srv1514373971:84 , srv1534293359:84 , srv1605405473:84 , srv1783502614:84 , srv1800411261:84 , srv1813569170:84 , srv1834391999:84 , srv1848614216:84 , srv1849610312:84 , srv1869290169:84 , srv1959352068:84 , srv1990354980:84 , srv2041638469:84 , srv2042544588:84 , srv2064174110:84 , srv2070300908:84 , srv2092112688:84 , srv212233524:84 , srv2133828072:84 , srv361681285:84 , srv369559327:84 , srv399147516:84 , srv406064715:84 , srv562906435:84 , srv759506360:84 , srv843712502:84 , srv925497113:84 , srv976279726:84 , srv100409858:85 , srv104226524:85 , srv110395153:85 , srv1104174171:85 , srv1109853382:85 , srv112096380:85 , srv1121627461:85 , srv1194905691:85 , srv1248941656:85 , srv1259572104:85 , srv1302066959:85 , srv1306980690:85 , srv1333421896:85 , srv1478762778:85 , srv1480098103:85 , srv1488932493:85 , srv1551668639:85 , srv1581592896:85 , srv167393982:85 , srv1726185166:85 , srv178634301:85 , srv1803766102:85 , srv1827084454:85 , srv1832540137:85 , srv1866343594:85 , srv1868218152:85 , srv1918959452:85 , srv1971150344:85 , srv2052032218:85 , srv2137680770:85 , srv235049208:85 , srv238467075:85 , srv249724131:85 , srv294717339:85 , srv372979765:85 , srv389220266:85 , srv390471424:85 , srv458698055:85 , srv460651088:85 , srv490720432:85 , srv566140504:85 , srv592188157:85 , srv664673092:85 , srv682617228:85 , srv700189506:85 , srv719234457:85 , srv766958575:85 , srv773148327:85 , srv863221191:85 , srv896824458:85 , srv961435309:85 , srv966106435:85 , srv1440508619:86 , srv288377538:86 , srv342012692:86 , srv52162442:86 , srv1208005277:626 }
2023-11-11 04:55:18,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:18,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv566140504=83, srv966106435=98, srv896824458=95, srv843712502=93, srv1485555977=30, srv1534293359=33, srv1255959885=11, srv389220266=74, srv1803766102=42, srv1184481417=7, srv773148327=92, srv766958575=91, srv1259572104=12, srv1971150344=54, srv1551668639=34, srv2092112688=61, srv1273803934=13, srv682617228=87, srv1398025435=23, srv1194905691=8, srv1297589107=14, srv562906435=82, srv1848614216=47, srv961435309=97, srv361681285=71, srv1918959452=52, srv1478762778=28, srv369559327=72, srv2041638469=56, srv249724131=67, srv212233524=62, srv976279726=99, srv1306980690=17, srv2133828072=63, srv759506360=90, srv178634301=40, srv238467075=66, srv104226524=1, srv1832540137=45, srv863221191=94, srv1488932493=31, srv342012692=70, srv294717339=69, srv399147516=76, srv2042544588=57, srv1480098103=29, srv1959352068=53, srv1581592896=35, srv52162442=81, srv1866343594=49, srv700189506=88, srv1208005277=9, srv1849610312=48, srv64675221=85, srv458698055=78, srv1869290169=51, srv1471379363=27, srv235049208=65, srv664673092=86, srv1800411261=41, srv1300865843=15, srv1455579219=26, srv167393982=37, srv1302066959=16, srv112096380=5, srv406064715=77, srv110395153=2, srv2064174110=59, srv1378246477=20, srv460651088=79, srv1445796511=25, srv2137680770=64, srv1390759139=21, srv2052032218=58, srv1990354980=55, srv1868218152=50, srv390471424=75, srv1394910569=22, srv490720432=80, srv1104174171=3, srv1333421896=18, srv1248941656=10, srv1726185166=38, srv1109853382=4, srv2070300908=60, srv288377538=68, srv719234457=89, srv1605405473=36, srv1834391999=46, srv592188157=84, srv1783502614=39, srv372979765=73, srv100409858=0, srv1813569170=43, srv1440508619=24, srv1827084454=44, srv1121627461=6, srv1514373971=32, srv1365706583=19, srv925497113=96} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:18,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:18,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-11 04:55:18,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-11 04:55:18,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-11 04:55:18,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-11 04:55:18,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-11 04:55:18,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-11 04:55:18,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-11 04:55:18,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-11 04:55:18,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-11 04:55:18,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-11 04:55:18,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-11 04:55:18,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-11 04:55:18,712 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.965681556703379E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:18,712 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1394910569:83 , srv64675221:83 , srv1184481417:84 , srv1255959885:84 , srv1273803934:84 , srv1297589107:84 , srv1300865843:84 , srv1365706583:84 , srv1378246477:84 , srv1390759139:84 , srv1398025435:84 , srv1445796511:84 , srv1455579219:84 , srv1471379363:84 , srv1485555977:84 , srv1514373971:84 , srv1534293359:84 , srv1605405473:84 , srv1783502614:84 , srv1800411261:84 , srv1813569170:84 , srv1834391999:84 , srv1848614216:84 , srv1849610312:84 , srv1869290169:84 , srv1959352068:84 , srv1990354980:84 , srv2041638469:84 , srv2042544588:84 , srv2064174110:84 , srv2070300908:84 , srv2092112688:84 , srv212233524:84 , srv2133828072:84 , srv361681285:84 , srv369559327:84 , srv399147516:84 , srv406064715:84 , srv562906435:84 , srv759506360:84 , srv843712502:84 , srv925497113:84 , srv976279726:84 , srv100409858:85 , srv104226524:85 , srv110395153:85 , srv1104174171:85 , srv1109853382:85 , srv112096380:85 , srv1121627461:85 , srv1194905691:85 , srv1248941656:85 , srv1259572104:85 , srv1302066959:85 , srv1306980690:85 , srv1333421896:85 , srv1478762778:85 , srv1480098103:85 , srv1488932493:85 , srv1551668639:85 , srv1581592896:85 , srv167393982:85 , srv1726185166:85 , srv178634301:85 , srv1803766102:85 , srv1827084454:85 , srv1832540137:85 , srv1866343594:85 , srv1868218152:85 , srv1918959452:85 , srv1971150344:85 , srv2052032218:85 , srv2137680770:85 , srv235049208:85 , srv238467075:85 , srv249724131:85 , srv294717339:85 , srv372979765:85 , srv389220266:85 , srv390471424:85 , srv458698055:85 , srv460651088:85 , srv490720432:85 , srv566140504:85 , srv592188157:85 , srv664673092:85 , srv682617228:85 , srv700189506:85 , srv719234457:85 , srv766958575:85 , srv773148327:85 , srv863221191:85 , srv896824458:85 , srv961435309:85 , srv966106435:85 , srv1440508619:86 , srv288377538:86 , srv342012692:86 , srv52162442:86 , srv1208005277:626 }
2023-11-11 04:55:18,740 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=200 (was 200), ProcessCount=167 (was 167), AvailableMemoryMB=2442 (was 2932)
2023-11-11 04:55:18,747 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=200, ProcessCount=167, AvailableMemoryMB=2442
2023-11-11 04:55:18,748 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-11 04:55:18,748 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-11 04:55:18,760 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1077493376:84 , srv117169993:84 , srv1286588012:84 , srv1335207725:84 , srv1358311429:84 , srv1511020268:84 , srv1527529896:84 , srv1641822777:84 , srv166185526:84 , srv1815670555:84 , srv1883764230:84 , srv1952024428:84 , srv200670086:84 , srv2038746354:84 , srv2096791123:84 , srv2144615482:84 , srv301279356:84 , srv4170656:84 , srv594154233:84 , srv632011813:84 , srv636841202:84 , srv647566962:84 , srv709463535:84 , srv726970461:84 , srv757424829:84 , srv803215094:84 , srv919953336:84 , srv968156610:84 , srv979312797:84 , srv1684244343:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-11 04:55:18,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:55:18,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1952024428=12, srv2096791123=15, srv301279356=17, srv117169993=1, srv1511020268=5, srv594154233=19, srv2038746354=14, srv2144615482=16, srv1286588012=2, srv709463535=23, srv166185526=8, srv757424829=25, srv1815670555=10, srv200670086=13, srv1358311429=4, srv968156610=28, srv1527529896=6, srv726970461=24, srv1883764230=11, srv647566962=22, srv1077493376=0, srv4170656=18, srv1641822777=7, srv803215094=26, srv1335207725=3, srv1684244343=9, srv979312797=29, srv919953336=27, srv632011813=20, srv636841202=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:55:18,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:55:18,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:55:18,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-11 04:55:18,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1303289151444178, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:55:18,790 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912603705420269, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1303289151444178, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-11 04:57:18,777 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6199281 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912603705420269 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:57:18,785 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1077493376:90 , srv117169993:90 , srv1286588012:90 , srv1335207725:90 , srv1358311429:90 , srv1511020268:90 , srv1527529896:90 , srv1641822777:90 , srv166185526:90 , srv1684244343:90 , srv1815670555:90 , srv1883764230:90 , srv1952024428:90 , srv200670086:90 , srv2038746354:90 , srv2096791123:90 , srv2144615482:90 , srv301279356:90 , srv4170656:90 , srv594154233:90 , srv632011813:90 , srv636841202:90 , srv647566962:90 , srv709463535:90 , srv726970461:90 , srv757424829:90 , srv803215094:90 , srv919953336:90 , srv968156610:90 , srv979312797:90 }
2023-11-11 04:57:18,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-11 04:57:18,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1952024428=12, srv2096791123=15, srv301279356=17, srv117169993=1, srv1511020268=5, srv594154233=19, srv2038746354=14, srv2144615482=16, srv1286588012=2, srv709463535=23, srv166185526=8, srv757424829=25, srv1815670555=10, srv200670086=13, srv1358311429=4, srv968156610=28, srv1527529896=6, srv726970461=24, srv1883764230=11, srv647566962=22, srv1077493376=0, srv4170656=18, srv1641822777=7, srv803215094=26, srv1335207725=3, srv1684244343=9, srv979312797=29, srv919953336=27, srv632011813=20, srv636841202=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-11 04:57:18,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-11 04:57:18,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-11 04:57:18,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-11 04:57:18,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-11 04:57:18,813 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-11 04:57:18,813 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1077493376:90 , srv117169993:90 , srv1286588012:90 , srv1335207725:90 , srv1358311429:90 , srv1511020268:90 , srv1527529896:90 , srv1641822777:90 , srv166185526:90 , srv1684244343:90 , srv1815670555:90 , srv1883764230:90 , srv1952024428:90 , srv200670086:90 , srv2038746354:90 , srv2096791123:90 , srv2144615482:90 , srv301279356:90 , srv4170656:90 , srv594154233:90 , srv632011813:90 , srv636841202:90 , srv647566962:90 , srv709463535:90 , srv726970461:90 , srv757424829:90 , srv803215094:90 , srv919953336:90 , srv968156610:90 , srv979312797:90 }
2023-11-11 04:57:18,829 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=213 (was 200) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=2501 (was 2442) - AvailableMemoryMB LEAK? -