2023-08-31 05:53:11,748 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-31 05:53:12,064 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-31 05:53:12,178 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-31 05:53:12,242 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 05:53:12,251 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-31 05:53:12,267 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=252, ProcessCount=170, AvailableMemoryMB=10774
2023-08-31 05:53:12,269 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 05:53:12,270 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-31 05:53:12,415 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001787363:84 , srv1004889679:84 , srv1037107176:84 , srv1070125550:84 , srv1076633695:84 , srv1096871424:84 , srv1108597679:84 , srv1119499805:84 , srv1141478451:84 , srv1147990960:84 , srv1151840771:84 , srv1167935900:84 , srv1185233574:84 , srv1188858744:84 , srv1197218277:84 , srv1206679007:84 , srv1207408249:84 , srv1236245145:84 , srv1260386320:84 , srv1262944729:84 , srv1272803015:84 , srv1291016943:84 , srv1321582919:84 , srv1326457298:84 , srv1326490416:84 , srv1362716652:84 , srv1388493098:84 , srv13939468:84 , srv1416544408:84 , srv141791115:84 , srv1425891398:84 , srv1445850800:84 , srv1452827114:84 , srv1506891976:84 , srv1515386081:84 , srv1519578340:84 , srv1562747638:84 , srv1564308794:84 , srv1636103331:84 , srv1651993870:84 , srv1662997507:84 , srv1680366540:84 , srv1680376464:84 , srv1685483497:84 , srv1685664431:84 , srv1698213091:84 , srv171298224:84 , srv1717492298:84 , srv1720591984:84 , srv173915674:84 , srv1740591032:84 , srv1770746845:84 , srv1823974653:84 , srv1831909141:84 , srv1856506488:84 , srv1929599201:84 , srv1942798869:84 , srv1973125548:84 , srv1977995964:84 , srv1982908199:84 , srv1996631595:84 , srv2063070544:84 , srv2065788853:84 , srv2076123137:84 , srv2105286984:84 , srv2124901345:84 , srv225831620:84 , srv23561449:84 , srv24094440:84 , srv241076829:84 , srv242805179:84 , srv246007400:84 , srv257548808:84 , srv272964563:84 , srv308364287:84 , srv325365869:84 , srv386407142:84 , srv389032820:84 , srv403714628:84 , srv407351641:84 , srv409498167:84 , srv473451665:84 , srv495657188:84 , srv509688020:84 , srv619284816:84 , srv621167040:84 , srv653892825:84 , srv673761209:84 , srv698682764:84 , srv70504742:84 , srv708318578:84 , srv770497062:84 , srv806900916:84 , srv842890242:84 , srv857102664:84 , srv89251042:84 , srv897154890:84 , srv991108565:84 , srv992147918:84 , srv2064537416:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-31 05:53:12,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:12,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:12,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:12,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:12,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:12,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:12,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:12,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:12,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:12,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:12,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:12,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:12,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:12,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:12,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:12,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:12,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:12,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:12,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:12,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:12,744 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 05:53:12,745 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 05:53:13,126 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 578 ms to try 5000 different iterations.  Found a solution that moves 4681 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1778408151049579. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041095486459677366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05846137942607539, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5201111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041095486459677366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:13,144 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1562747638:81 , srv1685483497:81 , srv1831909141:81 , srv495657188:81 , srv698682764:81 , srv1291016943:82 , srv1519578340:82 , srv1564308794:82 , srv806900916:82 , srv1206679007:83 , srv1416544408:83 , srv1515386081:83 , srv1651993870:83 , srv1982908199:83 , srv2124901345:83 , srv386407142:83 , srv473451665:83 , srv857102664:83 , srv1262944729:84 , srv1326490416:84 , srv141791115:84 , srv1680366540:84 , srv1685664431:84 , srv1973125548:84 , srv23561449:84 , srv272964563:84 , srv1108597679:85 , srv1167935900:85 , srv1185233574:85 , srv1188858744:85 , srv1236245145:85 , srv173915674:85 , srv1942798869:85 , srv2065788853:85 , srv257548808:85 , srv389032820:85 , srv407351641:85 , srv509688020:85 , srv673761209:85 , srv770497062:85 , srv842890242:85 , srv1096871424:86 , srv1147990960:86 , srv1197218277:86 , srv1362716652:86 , srv1506891976:86 , srv1636103331:86 , srv1698213091:86 , srv1929599201:86 , srv1977995964:86 , srv2063070544:86 , srv325365869:86 , srv403714628:86 , srv409498167:86 , srv621167040:86 , srv89251042:86 , srv1070125550:87 , srv1076633695:87 , srv1445850800:87 , srv1662997507:87 , srv1680376464:87 , srv1717492298:87 , srv1996631595:87 , srv2105286984:87 , srv24094440:87 , srv308364287:87 , srv708318578:87 , srv1119499805:88 , srv1141478451:88 , srv1207408249:88 , srv1260386320:88 , srv1425891398:88 , srv1740591032:88 , srv1823974653:88 , srv225831620:88 , srv619284816:88 , srv653892825:88 , srv1272803015:89 , srv1321582919:89 , srv1326457298:89 , srv13939468:89 , srv1720591984:89 , srv1770746845:89 , srv241076829:89 , srv897154890:89 , srv991108565:89 , srv1001787363:90 , srv1004889679:90 , srv246007400:90 , srv1151840771:91 , srv1452827114:91 , srv70504742:91 , srv992147918:91 , srv1037107176:92 , srv1856506488:92 , srv242805179:92 , srv171298224:93 , srv2076123137:93 , srv1388493098:94 , srv2064537416:455 }
2023-08-31 05:53:13,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:13,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:13,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:13,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:13,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:13,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:13,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:13,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:13,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:13,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:13,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:13,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:13,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:13,191 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:13,192 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:13,193 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:13,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:13,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:13,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:13,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:13,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:13,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:13,384 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041095486459677366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05846137942607539, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041095486459677366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:13,384 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 05:53:13,385 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17780803804239967, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.041095486459677366, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05846137942607539, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.041095486459677366, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17283333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24158333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 05:53:13,743 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 515 ms to try 5000 different iterations.  Found a solution that moves 2657 regions; Going from a computed imbalance of 0.17780803804239967 to a new imbalance of 0.01328124376686895. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021711872608003044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03478327964999673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2952222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021711872608003044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:13,755 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1388493098:77 , srv2076123137:78 , srv1037107176:80 , srv1452827114:80 , srv171298224:80 , srv1856506488:80 , srv242805179:80 , srv246007400:80 , srv1001787363:81 , srv1151840771:82 , srv1272803015:82 , srv1070125550:83 , srv1119499805:83 , srv1260386320:83 , srv1321582919:83 , srv1425891398:83 , srv1506891976:83 , srv1720591984:83 , srv1996631595:83 , srv225831620:83 , srv653892825:83 , srv897154890:83 , srv1004889679:84 , srv1141478451:84 , srv1326457298:84 , srv13939468:84 , srv1445850800:84 , srv1680376464:84 , srv1740591032:84 , srv1770746845:84 , srv1823974653:84 , srv308364287:84 , srv619284816:84 , srv70504742:84 , srv991108565:84 , srv992147918:84 , srv1076633695:85 , srv1108597679:85 , srv1207408249:85 , srv1662997507:85 , srv1942798869:85 , srv2105286984:85 , srv241076829:85 , srv325365869:85 , srv621167040:85 , srv708318578:85 , srv89251042:85 , srv1096871424:86 , srv1147990960:86 , srv1167935900:86 , srv1185233574:86 , srv1698213091:86 , srv2063070544:86 , srv24094440:86 , srv257548808:86 , srv389032820:86 , srv1197218277:87 , srv1326490416:87 , srv1362716652:87 , srv1636103331:87 , srv1680366540:87 , srv1717492298:87 , srv173915674:87 , srv1977995964:87 , srv386407142:87 , srv409498167:87 , srv509688020:87 , srv673761209:87 , srv770497062:87 , srv1236245145:88 , srv1291016943:88 , srv1651993870:88 , srv1685664431:88 , srv2065788853:88 , srv2124901345:88 , srv473451665:88 , srv842890242:88 , srv857102664:88 , srv1188858744:89 , srv1564308794:89 , srv1685483497:89 , srv1929599201:89 , srv1973125548:89 , srv1982908199:89 , srv23561449:89 , srv272964563:89 , srv403714628:89 , srv698682764:89 , srv1206679007:90 , srv1262944729:90 , srv1416544408:90 , srv1515386081:90 , srv1519578340:90 , srv1831909141:90 , srv407351641:90 , srv141791115:91 , srv1562747638:91 , srv806900916:92 , srv495657188:93 , srv2064537416:512 }
2023-08-31 05:53:13,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:13,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:13,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:13,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:13,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:13,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:13,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:13,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:13,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:13,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:13,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:13,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:13,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:13,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:13,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021711872608003044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03478327964999673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021711872608003044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:13,962 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 05:53:13,962 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013262639055222474, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021711872608003044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03478327964999673, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021711872608003044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04441666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 05:53:14,344 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 533 ms to try 5000 different iterations.  Found a solution that moves 959 regions; Going from a computed imbalance of 0.013262639055222474 to a new imbalance of 8.220349990507633E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006659180233461053); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010667929218208838); MoveCostFunction : (multiplier=7.0, imbalance=0.10655555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006659180233461053); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:14,346 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1929599201:82 , srv403714628:82 , srv141791115:83 , srv171298224:83 , srv407351641:83 , srv495657188:83 , srv70504742:83 , srv1004889679:84 , srv1147990960:84 , srv1151840771:84 , srv1188858744:84 , srv1197218277:84 , srv1207408249:84 , srv1262944729:84 , srv1326457298:84 , srv1416544408:84 , srv1636103331:84 , srv1717492298:84 , srv1977995964:84 , srv2065788853:84 , srv23561449:84 , srv241076829:84 , srv272964563:84 , srv806900916:84 , srv842890242:84 , srv992147918:84 , srv1037107176:85 , srv1096871424:85 , srv1206679007:85 , srv1236245145:85 , srv1321582919:85 , srv1362716652:85 , srv13939468:85 , srv1515386081:85 , srv1564308794:85 , srv1662997507:85 , srv1698213091:85 , srv1720591984:85 , srv1740591032:85 , srv1770746845:85 , srv1856506488:85 , srv1973125548:85 , srv1982908199:85 , srv2105286984:85 , srv242805179:85 , srv409498167:85 , srv619284816:85 , srv708318578:85 , srv770497062:85 , srv89251042:85 , srv991108565:85 , srv1001787363:86 , srv1076633695:86 , srv1119499805:86 , srv1141478451:86 , srv1167935900:86 , srv1260386320:86 , srv1291016943:86 , srv1326490416:86 , srv1388493098:86 , srv1445850800:86 , srv1452827114:86 , srv1519578340:86 , srv1562747638:86 , srv1651993870:86 , srv1680376464:86 , srv1685664431:86 , srv173915674:86 , srv1823974653:86 , srv1831909141:86 , srv2063070544:86 , srv2124901345:86 , srv225831620:86 , srv24094440:86 , srv257548808:86 , srv325365869:86 , srv389032820:86 , srv473451665:86 , srv509688020:86 , srv653892825:86 , srv673761209:86 , srv897154890:86 , srv1070125550:87 , srv1108597679:87 , srv1185233574:87 , srv1272803015:87 , srv1680366540:87 , srv1942798869:87 , srv2076123137:87 , srv246007400:87 , srv308364287:87 , srv621167040:87 , srv698682764:87 , srv857102664:87 , srv1425891398:88 , srv1506891976:88 , srv1685483497:88 , srv1996631595:88 , srv386407142:88 , srv2064537416:550 }
2023-08-31 05:53:14,350 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:14,351 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:14,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:14,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:14,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:14,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006659180233461053); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010667929218208838); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006659180233461053); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:14,524 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 05:53:14,524 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.153199375268755E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006659180233461053); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010667929218208838); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006659180233461053); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004833333333333334); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 05:53:14,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 538 ms to try 5000 different iterations.  Found a solution that moves 271 regions; Going from a computed imbalance of 8.153199375268755E-4 to a new imbalance of 4.950390502281898E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.030111111111111113, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:14,935 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1425891398:82 , srv1076633695:83 , srv991108565:83 , srv992147918:83 , srv1004889679:84 , srv1108597679:84 , srv1119499805:84 , srv1141478451:84 , srv1188858744:84 , srv1206679007:84 , srv1236245145:84 , srv1272803015:84 , srv1362716652:84 , srv1388493098:84 , srv13939468:84 , srv141791115:84 , srv1515386081:84 , srv1519578340:84 , srv1636103331:84 , srv1680366540:84 , srv1685483497:84 , srv1685664431:84 , srv1717492298:84 , srv173915674:84 , srv1770746845:84 , srv1823974653:84 , srv1973125548:84 , srv1996631595:84 , srv2063070544:84 , srv2076123137:84 , srv24094440:84 , srv241076829:84 , srv308364287:84 , srv407351641:84 , srv409498167:84 , srv509688020:84 , srv621167040:84 , srv673761209:84 , srv698682764:84 , srv70504742:84 , srv806900916:84 , srv842890242:84 , srv857102664:84 , srv897154890:84 , srv1001787363:85 , srv1037107176:85 , srv1070125550:85 , srv1096871424:85 , srv1151840771:85 , srv1167935900:85 , srv1185233574:85 , srv1197218277:85 , srv1207408249:85 , srv1260386320:85 , srv1262944729:85 , srv1321582919:85 , srv1326457298:85 , srv1326490416:85 , srv1416544408:85 , srv1452827114:85 , srv1506891976:85 , srv1562747638:85 , srv1651993870:85 , srv1662997507:85 , srv1680376464:85 , srv1698213091:85 , srv171298224:85 , srv1720591984:85 , srv1740591032:85 , srv1831909141:85 , srv1856506488:85 , srv1929599201:85 , srv1942798869:85 , srv1977995964:85 , srv1982908199:85 , srv2065788853:85 , srv2105286984:85 , srv2124901345:85 , srv225831620:85 , srv242805179:85 , srv246007400:85 , srv272964563:85 , srv325365869:85 , srv386407142:85 , srv389032820:85 , srv403714628:85 , srv473451665:85 , srv495657188:85 , srv619284816:85 , srv653892825:85 , srv708318578:85 , srv1147990960:86 , srv1291016943:86 , srv1445850800:86 , srv1564308794:86 , srv23561449:86 , srv257548808:86 , srv770497062:86 , srv89251042:86 , srv2064537416:626 }
2023-08-31 05:53:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:14,941 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:14,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:14,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:14,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:14,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:14,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:14,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:14,964 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:14,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:14,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:14,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:15,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:15,123 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-31 05:53:15,130 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.7606322464973714E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-31 05:53:15,473 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 495 ms to try 5000 different iterations.  Found a solution that moves 28 regions; Going from a computed imbalance of 4.7606322464973714E-5 to a new imbalance of 6.940129244310425E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.003111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:15,474 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1185233574:83 , srv1262944729:83 , srv1445850800:83 , srv1562747638:83 , srv23561449:83 , srv24094440:83 , srv257548808:83 , srv386407142:83 , srv770497062:83 , srv1001787363:84 , srv1004889679:84 , srv1037107176:84 , srv1070125550:84 , srv1096871424:84 , srv1141478451:84 , srv1147990960:84 , srv1151840771:84 , srv1167935900:84 , srv1188858744:84 , srv1197218277:84 , srv1206679007:84 , srv1207408249:84 , srv1236245145:84 , srv1260386320:84 , srv1272803015:84 , srv1291016943:84 , srv1321582919:84 , srv1326457298:84 , srv1326490416:84 , srv1362716652:84 , srv13939468:84 , srv1416544408:84 , srv141791115:84 , srv1452827114:84 , srv1506891976:84 , srv1515386081:84 , srv1519578340:84 , srv1564308794:84 , srv1651993870:84 , srv1662997507:84 , srv1680366540:84 , srv1680376464:84 , srv1685483497:84 , srv1685664431:84 , srv1698213091:84 , srv1717492298:84 , srv1720591984:84 , srv173915674:84 , srv1740591032:84 , srv1770746845:84 , srv1823974653:84 , srv1831909141:84 , srv1856506488:84 , srv1929599201:84 , srv1942798869:84 , srv1973125548:84 , srv1977995964:84 , srv1982908199:84 , srv1996631595:84 , srv2063070544:84 , srv2065788853:84 , srv2076123137:84 , srv2105286984:84 , srv2124901345:84 , srv225831620:84 , srv241076829:84 , srv242805179:84 , srv246007400:84 , srv272964563:84 , srv308364287:84 , srv325365869:84 , srv389032820:84 , srv403714628:84 , srv407351641:84 , srv409498167:84 , srv473451665:84 , srv495657188:84 , srv509688020:84 , srv619284816:84 , srv621167040:84 , srv653892825:84 , srv673761209:84 , srv70504742:84 , srv708318578:84 , srv806900916:84 , srv857102664:84 , srv89251042:84 , srv897154890:84 , srv992147918:84 , srv1076633695:85 , srv1108597679:85 , srv1119499805:85 , srv1388493098:85 , srv1425891398:85 , srv1636103331:85 , srv171298224:85 , srv698682764:85 , srv842890242:85 , srv991108565:85 , srv2064537416:683 }
2023-08-31 05:53:15,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:15,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416544408=28, srv857102664=95, srv1515386081=34, srv2076123137=64, srv1070125550=3, srv1147990960=9, srv1119499805=7, srv1001787363=0, srv89251042=96, srv806900916=93, srv1720591984=48, srv1770746845=51, srv1004889679=1, srv1651993870=39, srv991108565=98, srv1562747638=36, srv403714628=79, srv770497062=92, srv509688020=84, srv1188858744=13, srv1831909141=53, srv2065788853=63, srv1388493098=26, srv1685664431=44, srv2105286984=65, srv1823974653=52, srv2064537416=62, srv257548808=73, srv409498167=81, srv698682764=89, srv1662997507=40, srv1506891976=33, srv225831620=67, srv1197218277=14, srv242805179=71, srv1698213091=45, srv673761209=88, srv1996631595=60, srv842890242=94, srv1519578340=35, srv1236245145=17, srv1326490416=24, srv1425891398=30, srv241076829=70, srv1272803015=20, srv1856506488=54, srv1037107176=2, srv1207408249=16, srv1973125548=57, srv2124901345=66, srv246007400=72, srv171298224=46, srv1096871424=5, srv621167040=86, srv1564308794=37, srv325365869=76, srv1942798869=56, srv1636103331=38, srv1362716652=25, srv1929599201=55, srv386407142=77, srv1326457298=23, srv141791115=29, srv1740591032=50, srv495657188=83, srv389032820=78, srv1321582919=22, srv1141478451=8, srv1260386320=18, srv1982908199=59, srv1206679007=15, srv173915674=49, srv1151840771=10, srv1076633695=4, srv407351641=80, srv708318578=91, srv1185233574=12, srv1167935900=11, srv1680376464=42, srv1291016943=21, srv1977995964=58, srv1262944729=19, srv1108597679=6, srv308364287=75, srv1717492298=47, srv2063070544=61, srv992147918=99, srv897154890=97, srv1680366540=41, srv23561449=68, srv70504742=90, srv272964563=74, srv24094440=69, srv1685483497=43, srv1445850800=31, srv13939468=27, srv653892825=87, srv1452827114=32, srv619284816=85, srv473451665=82} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:15,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:15,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-31 05:53:15,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-31 05:53:15,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-31 05:53:15,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-31 05:53:15,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-31 05:53:15,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-31 05:53:15,631 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:15,632 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1185233574:83 , srv1262944729:83 , srv1445850800:83 , srv1562747638:83 , srv23561449:83 , srv24094440:83 , srv257548808:83 , srv386407142:83 , srv770497062:83 , srv1001787363:84 , srv1004889679:84 , srv1037107176:84 , srv1070125550:84 , srv1096871424:84 , srv1141478451:84 , srv1147990960:84 , srv1151840771:84 , srv1167935900:84 , srv1188858744:84 , srv1197218277:84 , srv1206679007:84 , srv1207408249:84 , srv1236245145:84 , srv1260386320:84 , srv1272803015:84 , srv1291016943:84 , srv1321582919:84 , srv1326457298:84 , srv1326490416:84 , srv1362716652:84 , srv13939468:84 , srv1416544408:84 , srv141791115:84 , srv1452827114:84 , srv1506891976:84 , srv1515386081:84 , srv1519578340:84 , srv1564308794:84 , srv1651993870:84 , srv1662997507:84 , srv1680366540:84 , srv1680376464:84 , srv1685483497:84 , srv1685664431:84 , srv1698213091:84 , srv1717492298:84 , srv1720591984:84 , srv173915674:84 , srv1740591032:84 , srv1770746845:84 , srv1823974653:84 , srv1831909141:84 , srv1856506488:84 , srv1929599201:84 , srv1942798869:84 , srv1973125548:84 , srv1977995964:84 , srv1982908199:84 , srv1996631595:84 , srv2063070544:84 , srv2065788853:84 , srv2076123137:84 , srv2105286984:84 , srv2124901345:84 , srv225831620:84 , srv241076829:84 , srv242805179:84 , srv246007400:84 , srv272964563:84 , srv308364287:84 , srv325365869:84 , srv389032820:84 , srv403714628:84 , srv407351641:84 , srv409498167:84 , srv473451665:84 , srv495657188:84 , srv509688020:84 , srv619284816:84 , srv621167040:84 , srv653892825:84 , srv673761209:84 , srv70504742:84 , srv708318578:84 , srv806900916:84 , srv857102664:84 , srv89251042:84 , srv897154890:84 , srv992147918:84 , srv1076633695:85 , srv1108597679:85 , srv1119499805:85 , srv1388493098:85 , srv1425891398:85 , srv1636103331:85 , srv171298224:85 , srv698682764:85 , srv842890242:85 , srv991108565:85 , srv2064537416:683 }
2023-08-31 05:53:15,670 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=280 (was 252) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=9905 (was 10774)
2023-08-31 05:53:15,683 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=280, ProcessCount=170, AvailableMemoryMB=9904
2023-08-31 05:53:15,684 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-31 05:53:15,684 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-31 05:53:15,698 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1039948939:84 , srv1057998745:84 , srv111387946:84 , srv1175426829:84 , srv1213633401:84 , srv1233829570:84 , srv1275888328:84 , srv1563836332:84 , srv1565575948:84 , srv1977386683:84 , srv2020996056:84 , srv2039656814:84 , srv2072577766:84 , srv2086934153:84 , srv2114357490:84 , srv2141233037:84 , srv301918609:84 , srv37756709:84 , srv408245855:84 , srv421891238:84 , srv472991348:84 , srv481905517:84 , srv511565344:84 , srv598493935:84 , srv606982916:84 , srv614252111:84 , srv923622108:84 , srv941232338:84 , srv990695996:84 , srv678208793:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-31 05:53:15,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:53:15,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv472991348=20, srv1057998745=1, srv614252111=25, srv941232338=28, srv301918609=16, srv481905517=21, srv1213633401=4, srv2114357490=14, srv1233829570=5, srv1175426829=3, srv990695996=29, srv421891238=19, srv606982916=24, srv2020996056=10, srv1563836332=7, srv1275888328=6, srv1039948939=0, srv2086934153=13, srv1977386683=9, srv2039656814=11, srv408245855=18, srv2072577766=12, srv511565344=22, srv678208793=26, srv598493935=23, srv37756709=17, srv2141233037=15, srv1565575948=8, srv923622108=27, srv111387946=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:53:15,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:53:15,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:53:15,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-31 05:53:15,736 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0842929283522906, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:53:15,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912458647529102, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0842929283522906, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-31 05:55:15,719 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5697880 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912458647529102 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:55:15,726 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1039948939:90 , srv1057998745:90 , srv111387946:90 , srv1175426829:90 , srv1213633401:90 , srv1233829570:90 , srv1275888328:90 , srv1563836332:90 , srv1565575948:90 , srv1977386683:90 , srv2020996056:90 , srv2039656814:90 , srv2072577766:90 , srv2086934153:90 , srv2114357490:90 , srv2141233037:90 , srv301918609:90 , srv37756709:90 , srv408245855:90 , srv421891238:90 , srv472991348:90 , srv481905517:90 , srv511565344:90 , srv598493935:90 , srv606982916:90 , srv614252111:90 , srv678208793:90 , srv923622108:90 , srv941232338:90 , srv990695996:90 }
2023-08-31 05:55:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-31 05:55:15,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv472991348=20, srv1057998745=1, srv614252111=25, srv941232338=28, srv301918609=16, srv481905517=21, srv1213633401=4, srv2114357490=14, srv1233829570=5, srv1175426829=3, srv990695996=29, srv421891238=19, srv606982916=24, srv2020996056=10, srv1563836332=7, srv1275888328=6, srv1039948939=0, srv2086934153=13, srv1977386683=9, srv2039656814=11, srv408245855=18, srv2072577766=12, srv511565344=22, srv678208793=26, srv598493935=23, srv37756709=17, srv2141233037=15, srv1565575948=8, srv923622108=27, srv111387946=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-31 05:55:15,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-31 05:55:15,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-31 05:55:15,745 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-31 05:55:15,745 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1039948939:90 , srv1057998745:90 , srv111387946:90 , srv1175426829:90 , srv1213633401:90 , srv1233829570:90 , srv1275888328:90 , srv1563836332:90 , srv1565575948:90 , srv1977386683:90 , srv2020996056:90 , srv2039656814:90 , srv2072577766:90 , srv2086934153:90 , srv2114357490:90 , srv2141233037:90 , srv301918609:90 , srv37756709:90 , srv408245855:90 , srv421891238:90 , srv472991348:90 , srv481905517:90 , srv511565344:90 , srv598493935:90 , srv606982916:90 , srv614252111:90 , srv678208793:90 , srv923622108:90 , srv941232338:90 , srv990695996:90 }
2023-08-31 05:55:15,758 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 280), ProcessCount=168 (was 170), AvailableMemoryMB=9717 (was 9904)