2023-09-26 06:53:14,561 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-26 06:53:14,889 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-26 06:53:14,999 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-26 06:53:15,060 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-26 06:53:15,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-26 06:53:15,084 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=172, AvailableMemoryMB=6333
2023-09-26 06:53:15,086 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-26 06:53:15,087 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-26 06:53:15,230 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1058707720:84 , srv1077068088:84 , srv1127514518:84 , srv1128532530:84 , srv1134053269:84 , srv1140977781:84 , srv1187310397:84 , srv1212564811:84 , srv1213158607:84 , srv1245702308:84 , srv1253754827:84 , srv1254101551:84 , srv1261382954:84 , srv1273838397:84 , srv1286819269:84 , srv1289771916:84 , srv1294296728:84 , srv1328900102:84 , srv1368044169:84 , srv1369510312:84 , srv1373768739:84 , srv1382181121:84 , srv1382782048:84 , srv1404816378:84 , srv1427509942:84 , srv1514091847:84 , srv1564968489:84 , srv1572007261:84 , srv1574442779:84 , srv159082564:84 , srv1614379407:84 , srv1620710563:84 , srv1624587157:84 , srv1650792537:84 , srv1651579266:84 , srv1693544576:84 , srv1706107257:84 , srv170647321:84 , srv1712787215:84 , srv1730750897:84 , srv1743894666:84 , srv1780293661:84 , srv1794503935:84 , srv1799609608:84 , srv1811587190:84 , srv1899986167:84 , srv1931446953:84 , srv1939843563:84 , srv1987246652:84 , srv202694899:84 , srv2035618502:84 , srv2055959495:84 , srv2056369981:84 , srv2119607757:84 , srv2130893320:84 , srv22160731:84 , srv238459395:84 , srv249936145:84 , srv251966398:84 , srv267316451:84 , srv278399125:84 , srv287340689:84 , srv300408356:84 , srv304043913:84 , srv341334906:84 , srv344047330:84 , srv368630171:84 , srv392815099:84 , srv39525472:84 , srv426110331:84 , srv457992064:84 , srv470631380:84 , srv500126846:84 , srv508468861:84 , srv51232593:84 , srv537664492:84 , srv542190250:84 , srv559717842:84 , srv566942058:84 , srv56839609:84 , srv573362338:84 , srv593895753:84 , srv614449534:84 , srv643410243:84 , srv645786374:84 , srv645881543:84 , srv708177209:84 , srv75963254:84 , srv785751251:84 , srv809589395:84 , srv825808184:84 , srv830214034:84 , srv845587220:84 , srv895074749:84 , srv903652288:84 , srv923029753:84 , srv933354756:84 , srv946961129:84 , srv97269247:84 , srv1793241078:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-26 06:53:15,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:15,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1624587157=32, srv1564968489=26, srv1373768739=20, srv500126846=73, srv1328900102=17, srv1651579266=34, srv1793241078=42, srv1931446953=47, srv251966398=59, srv51232593=75, srv1987246652=49, srv1286819269=14, srv22160731=56, srv1140977781=5, srv1212564811=7, srv845587220=93, srv1811587190=45, srv278399125=61, srv457992064=71, srv1382782048=22, srv566942058=79, srv1077068088=1, srv1514091847=25, srv1253754827=10, srv933354756=97, srv1650792537=33, srv368630171=67, srv895074749=94, srv573362338=81, srv785751251=89, srv1261382954=12, srv170647321=37, srv643410243=84, srv1368044169=18, srv542190250=77, srv1743894666=40, srv1572007261=27, srv2035618502=51, srv344047330=66, srv341334906=65, srv1427509942=24, srv1712787215=38, srv1780293661=41, srv75963254=88, srv508468861=74, srv1693544576=35, srv1799609608=44, srv426110331=70, srv946961129=98, srv1706107257=36, srv249936145=58, srv392815099=68, srv614449534=83, srv1294296728=16, srv159082564=29, srv2055959495=52, srv559717842=78, srv1289771916=15, srv2056369981=53, srv300408356=63, srv1273838397=13, srv593895753=82, srv645881543=86, srv2130893320=55, srv1899986167=46, srv56839609=80, srv1404816378=23, srv903652288=95, srv1254101551=11, srv238459395=57, srv1128532530=3, srv1794503935=43, srv1574442779=28, srv1939843563=48, srv1245702308=9, srv923029753=96, srv825808184=91, srv830214034=92, srv645786374=85, srv1187310397=6, srv1134053269=4, srv1213158607=8, srv1382181121=21, srv287340689=62, srv2119607757=54, srv97269247=99, srv809589395=90, srv39525472=69, srv1369510312=19, srv1127514518=2, srv304043913=64, srv1620710563=31, srv1614379407=30, srv537664492=76, srv1058707720=0, srv202694899=50, srv267316451=60, srv470631380=72, srv708177209=87, srv1730750897=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:15,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:15,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:15,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:15,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:15,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:15,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:15,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-26 06:53:15,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-26 06:53:15,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-26 06:53:15,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-26 06:53:15,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-26 06:53:15,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-26 06:53:15,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-26 06:53:15,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-26 06:53:15,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-26 06:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-26 06:53:15,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-26 06:53:15,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-26 06:53:15,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-26 06:53:15,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-26 06:53:15,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-26 06:53:15,515 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:15,516 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-26 06:53:15,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-26 06:53:15,877 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 523 ms to try 5000 different iterations.  Found a solution that moves 4675 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18301008595315554. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039444009735041544, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05194532219860127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5194444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039444009735041544, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17866666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:15,887 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1212564811:80 , srv170647321:81 , srv1254101551:82 , srv1651579266:82 , srv1939843563:82 , srv593895753:82 , srv1140977781:83 , srv1373768739:83 , srv1574442779:83 , srv1693544576:83 , srv1743894666:83 , srv1780293661:83 , srv1799609608:83 , srv500126846:83 , srv946961129:83 , srv97269247:83 , srv1261382954:84 , srv1514091847:84 , srv159082564:84 , srv1712787215:84 , srv1899986167:84 , srv2130893320:84 , srv392815099:84 , srv457992064:84 , srv51232593:84 , srv56839609:84 , srv645881543:84 , srv708177209:84 , srv933354756:84 , srv1077068088:85 , srv1273838397:85 , srv1404816378:85 , srv1572007261:85 , srv1620710563:85 , srv1987246652:85 , srv278399125:85 , srv426110331:85 , srv537664492:85 , srv1058707720:86 , srv1128532530:86 , srv1213158607:86 , srv1253754827:86 , srv1624587157:86 , srv1650792537:86 , srv1706107257:86 , srv1730750897:86 , srv1794503935:86 , srv251966398:86 , srv542190250:86 , srv614449534:86 , srv785751251:86 , srv809589395:86 , srv923029753:86 , srv1134053269:87 , srv1187310397:87 , srv1289771916:87 , srv1328900102:87 , srv1382181121:87 , srv1614379407:87 , srv1811587190:87 , srv2055959495:87 , srv2056369981:87 , srv238459395:87 , srv249936145:87 , srv267316451:87 , srv39525472:87 , srv566942058:87 , srv573362338:87 , srv75963254:87 , srv845587220:87 , srv1245702308:88 , srv1368044169:88 , srv1369510312:88 , srv1427509942:88 , srv1931446953:88 , srv2035618502:88 , srv287340689:88 , srv300408356:88 , srv470631380:88 , srv645786374:88 , srv825808184:88 , srv830214034:88 , srv1564968489:89 , srv304043913:89 , srv341334906:89 , srv368630171:89 , srv508468861:89 , srv903652288:89 , srv1127514518:90 , srv1294296728:90 , srv559717842:90 , srv22160731:91 , srv1286819269:92 , srv344047330:93 , srv895074749:93 , srv2119607757:95 , srv1382782048:97 , srv202694899:97 , srv643410243:97 , srv1793241078:440 }
2023-09-26 06:53:15,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:15,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1624587157=32, srv1564968489=26, srv1373768739=20, srv500126846=73, srv1328900102=17, srv1651579266=34, srv1793241078=42, srv1931446953=47, srv251966398=59, srv51232593=75, srv1987246652=49, srv1286819269=14, srv22160731=56, srv1140977781=5, srv1212564811=7, srv845587220=93, srv1811587190=45, srv278399125=61, srv457992064=71, srv1382782048=22, srv566942058=79, srv1077068088=1, srv1514091847=25, srv1253754827=10, srv933354756=97, srv1650792537=33, srv368630171=67, srv895074749=94, srv573362338=81, srv785751251=89, srv1261382954=12, srv170647321=37, srv643410243=84, srv1368044169=18, srv542190250=77, srv1743894666=40, srv1572007261=27, srv2035618502=51, srv344047330=66, srv341334906=65, srv1427509942=24, srv1712787215=38, srv1780293661=41, srv75963254=88, srv508468861=74, srv1693544576=35, srv1799609608=44, srv426110331=70, srv946961129=98, srv1706107257=36, srv249936145=58, srv392815099=68, srv614449534=83, srv1294296728=16, srv159082564=29, srv2055959495=52, srv559717842=78, srv1289771916=15, srv2056369981=53, srv300408356=63, srv1273838397=13, srv593895753=82, srv645881543=86, srv2130893320=55, srv1899986167=46, srv56839609=80, srv1404816378=23, srv903652288=95, srv1254101551=11, srv238459395=57, srv1128532530=3, srv1794503935=43, srv1574442779=28, srv1939843563=48, srv1245702308=9, srv923029753=96, srv825808184=91, srv830214034=92, srv645786374=85, srv1187310397=6, srv1134053269=4, srv1213158607=8, srv1382181121=21, srv287340689=62, srv2119607757=54, srv97269247=99, srv809589395=90, srv39525472=69, srv1369510312=19, srv1127514518=2, srv304043913=64, srv1620710563=31, srv1614379407=30, srv537664492=76, srv1058707720=0, srv202694899=50, srv267316451=60, srv470631380=72, srv708177209=87, srv1730750897=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:15,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:15,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:15,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:15,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:15,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-26 06:53:15,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-26 06:53:15,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-26 06:53:15,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-26 06:53:15,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-26 06:53:15,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-26 06:53:15,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-26 06:53:15,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-26 06:53:15,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-26 06:53:16,066 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039444009735041544, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05194532219860127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039444009735041544, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17866666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:16,066 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-26 06:53:16,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18297735090349532, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039444009735041544, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05194532219860127, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039444009735041544, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17866666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24108333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-26 06:53:16,501 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 562 ms to try 5000 different iterations.  Found a solution that moves 2665 regions; Going from a computed imbalance of 0.18297735090349532 to a new imbalance of 0.015536428519507511. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01948904616864718); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029814239699997195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2961111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01948904616864718); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:16,515 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1382782048:76 , srv202694899:76 , srv2119607757:76 , srv643410243:76 , srv1294296728:80 , srv344047330:80 , srv559717842:80 , srv1286819269:81 , srv895074749:81 , srv1127514518:82 , srv1564968489:82 , srv22160731:82 , srv1931446953:83 , srv238459395:83 , srv304043913:83 , srv368630171:83 , srv830214034:83 , srv1058707720:84 , srv1368044169:84 , srv1369510312:84 , srv1572007261:84 , srv1730750897:84 , srv1811587190:84 , srv2055959495:84 , srv287340689:84 , srv341334906:84 , srv566942058:84 , srv645786374:84 , srv903652288:84 , srv1382181121:85 , srv1514091847:85 , srv1614379407:85 , srv1624587157:85 , srv1706107257:85 , srv39525472:85 , srv426110331:85 , srv470631380:85 , srv508468861:85 , srv542190250:85 , srv573362338:85 , srv923029753:85 , srv1187310397:86 , srv1213158607:86 , srv1245702308:86 , srv1794503935:86 , srv2035618502:86 , srv2056369981:86 , srv251966398:86 , srv300408356:86 , srv392815099:86 , srv537664492:86 , srv708177209:86 , srv75963254:86 , srv825808184:86 , srv845587220:86 , srv1128532530:87 , srv1134053269:87 , srv1253754827:87 , srv1261382954:87 , srv1328900102:87 , srv1373768739:87 , srv1712787215:87 , srv278399125:87 , srv785751251:87 , srv809589395:87 , srv1077068088:88 , srv1254101551:88 , srv1273838397:88 , srv1427509942:88 , srv1620710563:88 , srv1650792537:88 , srv1780293661:88 , srv1899986167:88 , srv249936145:88 , srv267316451:88 , srv56839609:88 , srv614449534:88 , srv645881543:88 , srv933354756:88 , srv946961129:88 , srv1140977781:89 , srv1289771916:89 , srv1404816378:89 , srv1574442779:89 , srv159082564:89 , srv1743894666:89 , srv1939843563:89 , srv500126846:89 , srv51232593:89 , srv593895753:89 , srv1651579266:90 , srv170647321:90 , srv1799609608:90 , srv2130893320:90 , srv457992064:90 , srv97269247:90 , srv1212564811:91 , srv1693544576:91 , srv1987246652:92 , srv1793241078:507 }
2023-09-26 06:53:16,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:16,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1624587157=32, srv1564968489=26, srv1373768739=20, srv500126846=73, srv1328900102=17, srv1651579266=34, srv1793241078=42, srv1931446953=47, srv251966398=59, srv51232593=75, srv1987246652=49, srv1286819269=14, srv22160731=56, srv1140977781=5, srv1212564811=7, srv845587220=93, srv1811587190=45, srv278399125=61, srv457992064=71, srv1382782048=22, srv566942058=79, srv1077068088=1, srv1514091847=25, srv1253754827=10, srv933354756=97, srv1650792537=33, srv368630171=67, srv895074749=94, srv573362338=81, srv785751251=89, srv1261382954=12, srv170647321=37, srv643410243=84, srv1368044169=18, srv542190250=77, srv1743894666=40, srv1572007261=27, srv2035618502=51, srv344047330=66, srv341334906=65, srv1427509942=24, srv1712787215=38, srv1780293661=41, srv75963254=88, srv508468861=74, srv1693544576=35, srv1799609608=44, srv426110331=70, srv946961129=98, srv1706107257=36, srv249936145=58, srv392815099=68, srv614449534=83, srv1294296728=16, srv159082564=29, srv2055959495=52, srv559717842=78, srv1289771916=15, srv2056369981=53, srv300408356=63, srv1273838397=13, srv593895753=82, srv645881543=86, srv2130893320=55, srv1899986167=46, srv56839609=80, srv1404816378=23, srv903652288=95, srv1254101551=11, srv238459395=57, srv1128532530=3, srv1794503935=43, srv1574442779=28, srv1939843563=48, srv1245702308=9, srv923029753=96, srv825808184=91, srv830214034=92, srv645786374=85, srv1187310397=6, srv1134053269=4, srv1213158607=8, srv1382181121=21, srv287340689=62, srv2119607757=54, srv97269247=99, srv809589395=90, srv39525472=69, srv1369510312=19, srv1127514518=2, srv304043913=64, srv1620710563=31, srv1614379407=30, srv537664492=76, srv1058707720=0, srv202694899=50, srv267316451=60, srv470631380=72, srv708177209=87, srv1730750897=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:16,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:16,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-26 06:53:16,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-26 06:53:16,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-26 06:53:16,578 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-26 06:53:16,579 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-26 06:53:16,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-26 06:53:16,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-26 06:53:16,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-26 06:53:16,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-26 06:53:16,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-26 06:53:16,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-26 06:53:16,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-26 06:53:16,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-26 06:53:16,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-26 06:53:16,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01948904616864718); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029814239699997195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01948904616864718); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:16,749 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-26 06:53:16,749 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015517767790663757, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01948904616864718); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029814239699997195, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01948904616864718); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012583333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-26 06:53:17,084 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 475 ms to try 5000 different iterations.  Found a solution that moves 1004 regions; Going from a computed imbalance of 0.015517767790663757 to a new imbalance of 7.02269562937603E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023636267710147128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005254467395410082); MoveCostFunction : (multiplier=7.0, imbalance=0.11155555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0023636267710147128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:17,087 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1289771916:82 , srv1427509942:82 , srv1987246652:82 , srv249936145:82 , srv1328900102:83 , srv267316451:83 , srv1077068088:84 , srv1134053269:84 , srv1187310397:84 , srv1245702308:84 , srv1253754827:84 , srv1273838397:84 , srv1286819269:84 , srv1404816378:84 , srv1650792537:84 , srv1693544576:84 , srv2035618502:84 , srv2056369981:84 , srv300408356:84 , srv344047330:84 , srv457992064:84 , srv508468861:84 , srv809589395:84 , srv825808184:84 , srv845587220:84 , srv895074749:84 , srv903652288:84 , srv97269247:84 , srv1128532530:85 , srv1140977781:85 , srv1261382954:85 , srv1382181121:85 , srv1382782048:85 , srv159082564:85 , srv1614379407:85 , srv1743894666:85 , srv1794503935:85 , srv1799609608:85 , srv1899986167:85 , srv202694899:85 , srv2130893320:85 , srv287340689:85 , srv304043913:85 , srv341334906:85 , srv470631380:85 , srv573362338:85 , srv614449534:85 , srv643410243:85 , srv785751251:85 , srv933354756:85 , srv1127514518:86 , srv1213158607:86 , srv1368044169:86 , srv1369510312:86 , srv1564968489:86 , srv1574442779:86 , srv1620710563:86 , srv1651579266:86 , srv2055959495:86 , srv22160731:86 , srv251966398:86 , srv278399125:86 , srv368630171:86 , srv39525472:86 , srv51232593:86 , srv537664492:86 , srv542190250:86 , srv56839609:86 , srv593895753:86 , srv645786374:86 , srv645881543:86 , srv75963254:86 , srv1212564811:87 , srv1373768739:87 , srv1624587157:87 , srv1706107257:87 , srv170647321:87 , srv1712787215:87 , srv1780293661:87 , srv1939843563:87 , srv2119607757:87 , srv426110331:87 , srv500126846:87 , srv923029753:87 , srv946961129:87 , srv1058707720:88 , srv1254101551:88 , srv1294296728:88 , srv1514091847:88 , srv1811587190:88 , srv1931446953:88 , srv238459395:88 , srv392815099:88 , srv559717842:88 , srv566942058:88 , srv708177209:88 , srv830214034:88 , srv1572007261:89 , srv1730750897:89 , srv1793241078:531 }
2023-09-26 06:53:17,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:17,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1624587157=32, srv1564968489=26, srv1373768739=20, srv500126846=73, srv1328900102=17, srv1651579266=34, srv1793241078=42, srv1931446953=47, srv251966398=59, srv51232593=75, srv1987246652=49, srv1286819269=14, srv22160731=56, srv1140977781=5, srv1212564811=7, srv845587220=93, srv1811587190=45, srv278399125=61, srv457992064=71, srv1382782048=22, srv566942058=79, srv1077068088=1, srv1514091847=25, srv1253754827=10, srv933354756=97, srv1650792537=33, srv368630171=67, srv895074749=94, srv573362338=81, srv785751251=89, srv1261382954=12, srv170647321=37, srv643410243=84, srv1368044169=18, srv542190250=77, srv1743894666=40, srv1572007261=27, srv2035618502=51, srv344047330=66, srv341334906=65, srv1427509942=24, srv1712787215=38, srv1780293661=41, srv75963254=88, srv508468861=74, srv1693544576=35, srv1799609608=44, srv426110331=70, srv946961129=98, srv1706107257=36, srv249936145=58, srv392815099=68, srv614449534=83, srv1294296728=16, srv159082564=29, srv2055959495=52, srv559717842=78, srv1289771916=15, srv2056369981=53, srv300408356=63, srv1273838397=13, srv593895753=82, srv645881543=86, srv2130893320=55, srv1899986167=46, srv56839609=80, srv1404816378=23, srv903652288=95, srv1254101551=11, srv238459395=57, srv1128532530=3, srv1794503935=43, srv1574442779=28, srv1939843563=48, srv1245702308=9, srv923029753=96, srv825808184=91, srv830214034=92, srv645786374=85, srv1187310397=6, srv1134053269=4, srv1213158607=8, srv1382181121=21, srv287340689=62, srv2119607757=54, srv97269247=99, srv809589395=90, srv39525472=69, srv1369510312=19, srv1127514518=2, srv304043913=64, srv1620710563=31, srv1614379407=30, srv537664492=76, srv1058707720=0, srv202694899=50, srv267316451=60, srv470631380=72, srv708177209=87, srv1730750897=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:17,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:17,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-26 06:53:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-26 06:53:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-26 06:53:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-26 06:53:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-26 06:53:17,122 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-26 06:53:17,123 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-26 06:53:17,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-26 06:53:17,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-26 06:53:17,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-26 06:53:17,265 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023636267710147128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005254467395410082); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0023636267710147128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:17,265 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-26 06:53:17,266 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.952394046790178E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023636267710147128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005254467395410082); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0023636267710147128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0031666666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-26 06:53:17,671 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 531 ms to try 5000 different iterations.  Found a solution that moves 196 regions; Going from a computed imbalance of 6.952394046790178E-4 to a new imbalance of 1.0772668001353076E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.021777777777777778); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:17,672 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1368044169:83 , srv1620710563:83 , srv1730750897:83 , srv1799609608:83 , srv1811587190:83 , srv1931446953:83 , srv1987246652:83 , srv2119607757:83 , srv22160731:83 , srv500126846:83 , srv51232593:83 , srv566942058:83 , srv614449534:83 , srv645881543:83 , srv75963254:83 , srv830214034:83 , srv1058707720:84 , srv1127514518:84 , srv1128532530:84 , srv1212564811:84 , srv1213158607:84 , srv1245702308:84 , srv1253754827:84 , srv1254101551:84 , srv1289771916:84 , srv1294296728:84 , srv1369510312:84 , srv1382782048:84 , srv1427509942:84 , srv1572007261:84 , srv1574442779:84 , srv159082564:84 , srv1624587157:84 , srv1651579266:84 , srv1693544576:84 , srv170647321:84 , srv1712787215:84 , srv1780293661:84 , srv1899986167:84 , srv1939843563:84 , srv202694899:84 , srv2130893320:84 , srv238459395:84 , srv251966398:84 , srv267316451:84 , srv278399125:84 , srv287340689:84 , srv300408356:84 , srv341334906:84 , srv368630171:84 , srv392815099:84 , srv39525472:84 , srv457992064:84 , srv470631380:84 , srv508468861:84 , srv559717842:84 , srv56839609:84 , srv643410243:84 , srv645786374:84 , srv785751251:84 , srv825808184:84 , srv895074749:84 , srv923029753:84 , srv946961129:84 , srv1077068088:85 , srv1134053269:85 , srv1140977781:85 , srv1187310397:85 , srv1273838397:85 , srv1286819269:85 , srv1328900102:85 , srv1373768739:85 , srv1382181121:85 , srv1404816378:85 , srv1514091847:85 , srv1564968489:85 , srv1614379407:85 , srv1650792537:85 , srv1706107257:85 , srv1743894666:85 , srv1794503935:85 , srv2035618502:85 , srv2055959495:85 , srv2056369981:85 , srv249936145:85 , srv304043913:85 , srv344047330:85 , srv426110331:85 , srv537664492:85 , srv542190250:85 , srv573362338:85 , srv593895753:85 , srv708177209:85 , srv809589395:85 , srv845587220:85 , srv903652288:85 , srv933354756:85 , srv97269247:85 , srv1261382954:86 , srv1793241078:664 }
2023-09-26 06:53:17,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:17,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1624587157=32, srv1564968489=26, srv1373768739=20, srv500126846=73, srv1328900102=17, srv1651579266=34, srv1793241078=42, srv1931446953=47, srv251966398=59, srv51232593=75, srv1987246652=49, srv1286819269=14, srv22160731=56, srv1140977781=5, srv1212564811=7, srv845587220=93, srv1811587190=45, srv278399125=61, srv457992064=71, srv1382782048=22, srv566942058=79, srv1077068088=1, srv1514091847=25, srv1253754827=10, srv933354756=97, srv1650792537=33, srv368630171=67, srv895074749=94, srv573362338=81, srv785751251=89, srv1261382954=12, srv170647321=37, srv643410243=84, srv1368044169=18, srv542190250=77, srv1743894666=40, srv1572007261=27, srv2035618502=51, srv344047330=66, srv341334906=65, srv1427509942=24, srv1712787215=38, srv1780293661=41, srv75963254=88, srv508468861=74, srv1693544576=35, srv1799609608=44, srv426110331=70, srv946961129=98, srv1706107257=36, srv249936145=58, srv392815099=68, srv614449534=83, srv1294296728=16, srv159082564=29, srv2055959495=52, srv559717842=78, srv1289771916=15, srv2056369981=53, srv300408356=63, srv1273838397=13, srv593895753=82, srv645881543=86, srv2130893320=55, srv1899986167=46, srv56839609=80, srv1404816378=23, srv903652288=95, srv1254101551=11, srv238459395=57, srv1128532530=3, srv1794503935=43, srv1574442779=28, srv1939843563=48, srv1245702308=9, srv923029753=96, srv825808184=91, srv830214034=92, srv645786374=85, srv1187310397=6, srv1134053269=4, srv1213158607=8, srv1382181121=21, srv287340689=62, srv2119607757=54, srv97269247=99, srv809589395=90, srv39525472=69, srv1369510312=19, srv1127514518=2, srv304043913=64, srv1620710563=31, srv1614379407=30, srv537664492=76, srv1058707720=0, srv202694899=50, srv267316451=60, srv470631380=72, srv708177209=87, srv1730750897=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-26 06:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-26 06:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-26 06:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-26 06:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-26 06:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-26 06:53:17,816 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.40024666800374E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:17,817 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1368044169:83 , srv1620710563:83 , srv1730750897:83 , srv1799609608:83 , srv1811587190:83 , srv1931446953:83 , srv1987246652:83 , srv2119607757:83 , srv22160731:83 , srv500126846:83 , srv51232593:83 , srv566942058:83 , srv614449534:83 , srv645881543:83 , srv75963254:83 , srv830214034:83 , srv1058707720:84 , srv1127514518:84 , srv1128532530:84 , srv1212564811:84 , srv1213158607:84 , srv1245702308:84 , srv1253754827:84 , srv1254101551:84 , srv1289771916:84 , srv1294296728:84 , srv1369510312:84 , srv1382782048:84 , srv1427509942:84 , srv1572007261:84 , srv1574442779:84 , srv159082564:84 , srv1624587157:84 , srv1651579266:84 , srv1693544576:84 , srv170647321:84 , srv1712787215:84 , srv1780293661:84 , srv1899986167:84 , srv1939843563:84 , srv202694899:84 , srv2130893320:84 , srv238459395:84 , srv251966398:84 , srv267316451:84 , srv278399125:84 , srv287340689:84 , srv300408356:84 , srv341334906:84 , srv368630171:84 , srv392815099:84 , srv39525472:84 , srv457992064:84 , srv470631380:84 , srv508468861:84 , srv559717842:84 , srv56839609:84 , srv643410243:84 , srv645786374:84 , srv785751251:84 , srv825808184:84 , srv895074749:84 , srv923029753:84 , srv946961129:84 , srv1077068088:85 , srv1134053269:85 , srv1140977781:85 , srv1187310397:85 , srv1273838397:85 , srv1286819269:85 , srv1328900102:85 , srv1373768739:85 , srv1382181121:85 , srv1404816378:85 , srv1514091847:85 , srv1564968489:85 , srv1614379407:85 , srv1650792537:85 , srv1706107257:85 , srv1743894666:85 , srv1794503935:85 , srv2035618502:85 , srv2055959495:85 , srv2056369981:85 , srv249936145:85 , srv304043913:85 , srv344047330:85 , srv426110331:85 , srv537664492:85 , srv542190250:85 , srv573362338:85 , srv593895753:85 , srv708177209:85 , srv809589395:85 , srv845587220:85 , srv903652288:85 , srv933354756:85 , srv97269247:85 , srv1261382954:86 , srv1793241078:664 }
2023-09-26 06:53:17,841 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=334 (was 337), ProcessCount=172 (was 172), AvailableMemoryMB=5711 (was 6333)
2023-09-26 06:53:17,851 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=334, ProcessCount=172, AvailableMemoryMB=5710
2023-09-26 06:53:17,852 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-26 06:53:17,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-26 06:53:17,861 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1016855047:84 , srv1043166797:84 , srv1157191443:84 , srv1170796354:84 , srv120329243:84 , srv1208693703:84 , srv1391803121:84 , srv1416402994:84 , srv1549052902:84 , srv1595765032:84 , srv1596861819:84 , srv1642809209:84 , srv1659234145:84 , srv1685302370:84 , srv1711393855:84 , srv2130530492:84 , srv2138480267:84 , srv31667362:84 , srv384646083:84 , srv395657006:84 , srv4326182:84 , srv451685530:84 , srv55638609:84 , srv710413747:84 , srv735504359:84 , srv82529788:84 , srv830897191:84 , srv843503343:84 , srv84715069:84 , srv195885797:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-26 06:53:17,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:53:17,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1170796354=3, srv1595765032=9, srv1659234145=12, srv55638609=23, srv1043166797=1, srv384646083=19, srv31667362=18, srv84715069=29, srv1208693703=5, srv395657006=20, srv451685530=22, srv2138480267=17, srv1016855047=0, srv1391803121=6, srv195885797=15, srv4326182=21, srv1642809209=11, srv710413747=24, srv1685302370=13, srv1416402994=7, srv843503343=28, srv735504359=25, srv1549052902=8, srv1596861819=10, srv82529788=26, srv120329243=4, srv1157191443=2, srv1711393855=14, srv830897191=27, srv2130530492=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:53:17,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:53:17,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:53:17,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-26 06:53:17,889 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0898658166115758, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:53:17,889 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912476207518035, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0898658166115758, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-26 06:55:17,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5463889 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.9912476207518035 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:55:17,882 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1016855047:90 , srv1043166797:90 , srv1157191443:90 , srv1170796354:90 , srv120329243:90 , srv1208693703:90 , srv1391803121:90 , srv1416402994:90 , srv1549052902:90 , srv1595765032:90 , srv1596861819:90 , srv1642809209:90 , srv1659234145:90 , srv1685302370:90 , srv1711393855:90 , srv195885797:90 , srv2130530492:90 , srv2138480267:90 , srv31667362:90 , srv384646083:90 , srv395657006:90 , srv4326182:90 , srv451685530:90 , srv55638609:90 , srv710413747:90 , srv735504359:90 , srv82529788:90 , srv830897191:90 , srv843503343:90 , srv84715069:90 }
2023-09-26 06:55:17,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-26 06:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1170796354=3, srv1595765032=9, srv1659234145=12, srv55638609=23, srv1043166797=1, srv384646083=19, srv31667362=18, srv84715069=29, srv1208693703=5, srv395657006=20, srv451685530=22, srv2138480267=17, srv1016855047=0, srv1391803121=6, srv195885797=15, srv4326182=21, srv1642809209=11, srv710413747=24, srv1685302370=13, srv1416402994=7, srv843503343=28, srv735504359=25, srv1549052902=8, srv1596861819=10, srv82529788=26, srv120329243=4, srv1157191443=2, srv1711393855=14, srv830897191=27, srv2130530492=16} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-26 06:55:17,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-26 06:55:17,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-26 06:55:17,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-26 06:55:17,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-26 06:55:17,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-26 06:55:17,906 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-26 06:55:17,907 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1016855047:90 , srv1043166797:90 , srv1157191443:90 , srv1170796354:90 , srv120329243:90 , srv1208693703:90 , srv1391803121:90 , srv1416402994:90 , srv1549052902:90 , srv1595765032:90 , srv1596861819:90 , srv1642809209:90 , srv1659234145:90 , srv1685302370:90 , srv1711393855:90 , srv195885797:90 , srv2130530492:90 , srv2138480267:90 , srv31667362:90 , srv384646083:90 , srv395657006:90 , srv4326182:90 , srv451685530:90 , srv55638609:90 , srv710413747:90 , srv735504359:90 , srv82529788:90 , srv830897191:90 , srv843503343:90 , srv84715069:90 }
2023-09-26 06:55:17,919 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 334), ProcessCount=171 (was 172), AvailableMemoryMB=4601 (was 5710)